[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:12:50 dist-test-slave-1l3l krb5kdc[19701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:12:50 dist-test-slave-1l3l krb5kdc[19701](info): set up 2 sockets
Aug 08 02:12:50 dist-test-slave-1l3l krb5kdc[19701](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 02:12:51.704360 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:12:51 dist-test-slave-1l3l krb5kdc[19701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:12:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:12:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:12:51.816598 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42265
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:42265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:51.895390 19718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:51.895584 19718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:51.895614 19718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:51.896761 19718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:12:51.896817 19718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:51.896838 19718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:51.896854 19718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:12:51.896871 19718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:12:51.898290 19718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:42265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:51.898518 19718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:51.898721 19718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:51.903630 19730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:51.903752 19718 server_base.cc:1034] running on GCE node
W20260808 02:12:51.903621 19729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:51.903607 19734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:51.904114 19718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:51.904484 19718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:51.905642 19718 hybrid_clock.cc:648] HybridClock initialized: now 1786155171905625 us; error 35 us; skew 500 ppm
Aug 08 02:12:51 dist-test-slave-1l3l krb5kdc[19701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:51.908031 19718 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:12:51.908514 19718 webserver.cc:460] Webserver started at http://127.19.58.254:45149/ using document root <none> and password file <none>
I20260808 02:12:51.908706 19718 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:51.908754 19718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:51.908881 19718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:51.909986 19718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "5300a916cb6941dfb4b6da68e0a63c54"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-1l3l"
server_key: "9109c8b9825e104019348fd0e21c0b63"
server_key_iv: "f9bfbc1eab0b6460dc7567487bdef066"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.910339 19718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "5300a916cb6941dfb4b6da68e0a63c54"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-1l3l"
server_key: "9109c8b9825e104019348fd0e21c0b63"
server_key_iv: "f9bfbc1eab0b6460dc7567487bdef066"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.911684 19718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 02:12:51.912441 19744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:51.912591 19718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:12:51.912673 19718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "5300a916cb6941dfb4b6da68e0a63c54"
format_stamp: "Formatted at 2026-08-08 02:12:51 on dist-test-slave-1l3l"
server_key: "9109c8b9825e104019348fd0e21c0b63"
server_key_iv: "f9bfbc1eab0b6460dc7567487bdef066"
server_key_version: "encryptionkey@0"
I20260808 02:12:51.912734 19718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:51.921599 19718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:51.923447 19718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:51.923573 19718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:51.927343 19718 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:42265
I20260808 02:12:51.927373 19824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:42265 every 8 connection(s)
I20260808 02:12:51.927676 19718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:12:51.928488 19825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:51.930625 19825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54: Bootstrap starting.
I20260808 02:12:51.930678 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 19718
I20260808 02:12:51.930781 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:12:51.931173 19825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:51.931382 19691 external_mini_cluster.cc:1428] Setting key bb23e293a8743a6a331ea5fac8362149
I20260808 02:12:51.931442 19825 log.cc:824] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:51.936165 19825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54: No bootstrap required, opened a new log
I20260808 02:12:51.938468 19825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5300a916cb6941dfb4b6da68e0a63c54" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42265 } }
I20260808 02:12:51.938592 19825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:51.938613 19825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5300a916cb6941dfb4b6da68e0a63c54, State: Initialized, Role: FOLLOWER
I20260808 02:12:51.938736 19825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5300a916cb6941dfb4b6da68e0a63c54" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42265 } }
I20260808 02:12:51.938807 19825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:12:51.938828 19825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:12:51.938874 19825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:51.939384 19825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5300a916cb6941dfb4b6da68e0a63c54" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42265 } }
I20260808 02:12:51.939481 19825 leader_election.cc:302] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5300a916cb6941dfb4b6da68e0a63c54; no voters:
I20260808 02:12:51.939635 19825 leader_election.cc:288] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:12:51.939675 19833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:51.939800 19833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [term 1 LEADER]: Becoming Leader. State: Replica: 5300a916cb6941dfb4b6da68e0a63c54, State: Running, Role: LEADER
I20260808 02:12:51.939915 19825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:12:51.939899 19833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5300a916cb6941dfb4b6da68e0a63c54" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42265 } }
I20260808 02:12:51.940284 19834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5300a916cb6941dfb4b6da68e0a63c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5300a916cb6941dfb4b6da68e0a63c54" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42265 } } }
I20260808 02:12:51.940387 19834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:51.940289 19835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5300a916cb6941dfb4b6da68e0a63c54. Latest consensus state: current_term: 1 leader_uuid: "5300a916cb6941dfb4b6da68e0a63c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5300a916cb6941dfb4b6da68e0a63c54" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42265 } } }
I20260808 02:12:51.940627 19835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54 [sys.catalog]: This master's current role is: LEADER
W20260808 02:12:51.941720 19855 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:12:51.941776 19855 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:12:51.941828 19840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:12:51.941988 19840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 02:12:51 dist-test-slave-1l3l krb5kdc[19701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:51.943500 19840 catalog_manager.cc:1347] Generated new cluster ID: 2eee87d659b143e6ac6c71bd5e459184
I20260808 02:12:51.943552 19840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:12:51.947549 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:51.932308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47250 (local address 127.19.58.254:42265)
0808 02:12:51.932642 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:51.932649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:51.932675 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:12:51.934585 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:51.934591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:51.934633 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:51.934746 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:51.936352 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.937707 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.938687 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:51.939124 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:51.939220 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:51.943501 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:51.943505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:51.943505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:51.943515 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:51.945419 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.945926 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.945929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.945930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.946025 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:51.946290 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:51.946290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:51.946290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:51.946558 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:51.947159 (+ 601us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:51.947232 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 02:12:51.947352 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":132,"threads_started":1}
I20260808 02:12:51.951508 19840 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:12:51.952047 19840 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:12:51.975862 19840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5300a916cb6941dfb4b6da68e0a63c54: Generated new TSK 0
I20260808 02:12:51.976034 19840 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:12:52.006106 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42265
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:52.088450 19861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:52.088644 19861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:52.088675 19861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:52.089895 19861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:52.089960 19861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:52.089999 19861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:12:52.091621 19861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42265
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:52.091882 19861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:52.092084 19861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:52.096769 19876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:52.096822 19871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:52.096772 19879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:52.096868 19861 server_base.cc:1034] running on GCE node
I20260808 02:12:52.097199 19861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:52.097440 19861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:52.098575 19861 hybrid_clock.cc:648] HybridClock initialized: now 1786155172098560 us; error 34 us; skew 500 ppm
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[19701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:52.101120 19861 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:12:52.101531 19861 webserver.cc:460] Webserver started at http://127.19.58.193:35019/ using document root <none> and password file <none>
I20260808 02:12:52.101737 19861 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:52.101784 19861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:52.101907 19861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:52.102865 19861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "9a88e931171a4fadbeb5fe51fa76b556"
format_stamp: "Formatted at 2026-08-08 02:12:52 on dist-test-slave-1l3l"
server_key: "99bc58bc862174079bba00a160bcb0e5"
server_key_iv: "d1e596a4eed573e448d349a4432c1d0d"
server_key_version: "encryptionkey@0"
I20260808 02:12:52.103179 19861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "9a88e931171a4fadbeb5fe51fa76b556"
format_stamp: "Formatted at 2026-08-08 02:12:52 on dist-test-slave-1l3l"
server_key: "99bc58bc862174079bba00a160bcb0e5"
server_key_iv: "d1e596a4eed573e448d349a4432c1d0d"
server_key_version: "encryptionkey@0"
I20260808 02:12:52.104524 19861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:12:52.105170 19891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:52.105314 19861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:12:52.105420 19861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "9a88e931171a4fadbeb5fe51fa76b556"
format_stamp: "Formatted at 2026-08-08 02:12:52 on dist-test-slave-1l3l"
server_key: "99bc58bc862174079bba00a160bcb0e5"
server_key_iv: "d1e596a4eed573e448d349a4432c1d0d"
server_key_version: "encryptionkey@0"
I20260808 02:12:52.105480 19861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:52.114679 19861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:52.116765 19861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:52.116892 19861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:52.117087 19861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:52.117352 19861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:52.117393 19861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:52.117419 19861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:52.117443 19861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:52.123361 19861 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:40477
I20260808 02:12:52.123448 20063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:40477 every 8 connection(s)
I20260808 02:12:52.123723 19861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[19701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:52.131022 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 19861
I20260808 02:12:52.131119 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:12:52.131275 19691 external_mini_cluster.cc:1428] Setting key b3967296ac0b5e2db1902a8b4a969acf
I20260808 02:12:52.132293 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.124470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:46349 (local address 127.19.58.254:42265)
0808 02:12:52.124516 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.124518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.124689 (+ 171us) server_negotiation.cc:407] Connection header received
0808 02:12:52.125128 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.125129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.125137 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.125193 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:52.126114 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.126624 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.127533 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.127762 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.127810 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.129736 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:52.129737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:52.129737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:52.129743 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:52.130842 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.131265 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.131266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.131266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.131321 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.131614 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.131615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.131615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.131754 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:52.131809 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.132108 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.132200 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:12:52.132436 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.124421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42265 (local address 127.19.58.193:46349)
0808 02:12:52.124570 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.124609 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.125056 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.125242 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.125245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.125847 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:52.126029 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.126035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.126691 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.126692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.127457 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.127459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.127815 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.127816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.127910 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.128341 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:52.128349 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:52.129645 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:52.130913 (+ 1268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:52.130920 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:52.130922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:52.131202 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:52.131418 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:52.131419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:52.131420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:52.131557 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:52.131816 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:52.131822 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:52.131970 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.132074 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.132223 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260808 02:12:52.132846 20064 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42265
I20260808 02:12:52.132930 20064 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:52.133088 20064 heartbeater.cc:511] Master 127.19.58.254:42265 requested a full tablet report, sending...
I20260808 02:12:52.133567 19768 ts_manager.cc:194] Registered new tserver with Master: 9a88e931171a4fadbeb5fe51fa76b556 (127.19.58.193:40477)
I20260808 02:12:52.134419 19768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:46349
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:12:52.154664 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42265
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:52.234081 20070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:52.234272 20070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:52.234303 20070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:52.235773 20070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:52.235816 20070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:52.235849 20070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:12:52.237262 20070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42265
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:52.237486 20070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:52.237694 20070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:52.242574 20077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:52.242576 20081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:52.242576 20079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:52.242841 20070 server_base.cc:1034] running on GCE node
I20260808 02:12:52.243026 20070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:52.243247 20070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:52.244383 20070 hybrid_clock.cc:648] HybridClock initialized: now 1786155172244367 us; error 30 us; skew 500 ppm
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[19701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:52.246697 20070 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:12:52.247098 20070 webserver.cc:460] Webserver started at http://127.19.58.194:40765/ using document root <none> and password file <none>
I20260808 02:12:52.247292 20070 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:52.247339 20070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:52.247467 20070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:52.248474 20070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "c857b80aea4941209ab23b9fb56f54f5"
format_stamp: "Formatted at 2026-08-08 02:12:52 on dist-test-slave-1l3l"
server_key: "df65073e24d827e52f2dfe0fff2f1726"
server_key_iv: "43154c388fb84a72d87b05f8ab9bc421"
server_key_version: "encryptionkey@0"
I20260808 02:12:52.248793 20070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "c857b80aea4941209ab23b9fb56f54f5"
format_stamp: "Formatted at 2026-08-08 02:12:52 on dist-test-slave-1l3l"
server_key: "df65073e24d827e52f2dfe0fff2f1726"
server_key_iv: "43154c388fb84a72d87b05f8ab9bc421"
server_key_version: "encryptionkey@0"
I20260808 02:12:52.249966 20070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:12:52.250576 20088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:52.250720 20070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:12:52.250798 20070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "c857b80aea4941209ab23b9fb56f54f5"
format_stamp: "Formatted at 2026-08-08 02:12:52 on dist-test-slave-1l3l"
server_key: "df65073e24d827e52f2dfe0fff2f1726"
server_key_iv: "43154c388fb84a72d87b05f8ab9bc421"
server_key_version: "encryptionkey@0"
I20260808 02:12:52.250861 20070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:52.279661 20070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:52.281443 20070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:52.281551 20070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:52.281736 20070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:52.281981 20070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:52.282012 20070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:52.282032 20070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:52.282047 20070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:52.287919 20070 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:40379
I20260808 02:12:52.287978 20257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:40379 every 8 connection(s)
I20260808 02:12:52.288259 20070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:12:52.289299 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 20070
I20260808 02:12:52.289374 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:12:52.289525 19691 external_mini_cluster.cc:1428] Setting key f54f2d140ef20dcf0507d425d5053d0c
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[19701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
I20260808 02:12:52.298545 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.289009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:53999 (local address 127.19.58.254:42265)
0808 02:12:52.289064 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.289066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.289234 (+ 168us) server_negotiation.cc:407] Connection header received
0808 02:12:52.289790 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.289792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.289802 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.289872 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:52.290904 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.291658 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.293043 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.293292 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.293361 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.295882 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:52.295884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:52.295884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:52.295891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:52.297214 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.297660 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.297661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.297661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.297709 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.297962 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.297963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.297963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.298058 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:52.298103 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.298391 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.298456 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 02:12:52.298751 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.288957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42265 (local address 127.19.58.194:53999)
0808 02:12:52.289109 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.289149 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.289691 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.289943 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.289947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.290559 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:52.290795 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.290802 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.291788 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.291789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.292955 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.292959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.293437 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.293439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.293565 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.294118 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:52.294129 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:52.295792 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:52.297294 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:52.297301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:52.297303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:52.297589 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:52.297765 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:52.297766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:52.297767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:52.297897 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:52.298112 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:52.298119 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:52.298267 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.298370 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.298539 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260808 02:12:52.299173 20259 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42265
I20260808 02:12:52.299253 20259 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:52.299434 20259 heartbeater.cc:511] Master 127.19.58.254:42265 requested a full tablet report, sending...
I20260808 02:12:52.299801 19768 ts_manager.cc:194] Registered new tserver with Master: c857b80aea4941209ab23b9fb56f54f5 (127.19.58.194:40379)
I20260808 02:12:52.300170 19768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:53999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:12:52.314284 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42265
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:43013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:52.394238 20269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:52.394416 20269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:52.394438 20269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:52.395502 20269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:52.395537 20269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:52.395561 20269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:12:52.396978 20269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:43013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42265
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:52.397157 20269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:52.397320 20269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:12:52.401923 20269 server_base.cc:1034] running on GCE node
W20260808 02:12:52.401880 20285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:52.401883 20288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:52.401886 20281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:52.402218 20269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:52.402443 20269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:52.403580 20269 hybrid_clock.cc:648] HybridClock initialized: now 1786155172403563 us; error 30 us; skew 500 ppm
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[19701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:52.405931 20269 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:12:52.406335 20269 webserver.cc:460] Webserver started at http://127.19.58.195:35091/ using document root <none> and password file <none>
I20260808 02:12:52.406543 20269 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:52.406591 20269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:52.406713 20269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:52.407642 20269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "dbfe43701f4e4f609fa22f1065abb402"
format_stamp: "Formatted at 2026-08-08 02:12:52 on dist-test-slave-1l3l"
server_key: "b4a16d22f9576a7523d333ef8a18d2bc"
server_key_iv: "6eac765ecba13c67eb93d7c3a9ea1cff"
server_key_version: "encryptionkey@0"
I20260808 02:12:52.408008 20269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "dbfe43701f4e4f609fa22f1065abb402"
format_stamp: "Formatted at 2026-08-08 02:12:52 on dist-test-slave-1l3l"
server_key: "b4a16d22f9576a7523d333ef8a18d2bc"
server_key_iv: "6eac765ecba13c67eb93d7c3a9ea1cff"
server_key_version: "encryptionkey@0"
I20260808 02:12:52.409338 20269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:52.409979 20302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:52.410125 20269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:52.410202 20269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "dbfe43701f4e4f609fa22f1065abb402"
format_stamp: "Formatted at 2026-08-08 02:12:52 on dist-test-slave-1l3l"
server_key: "b4a16d22f9576a7523d333ef8a18d2bc"
server_key_iv: "6eac765ecba13c67eb93d7c3a9ea1cff"
server_key_version: "encryptionkey@0"
I20260808 02:12:52.410265 20269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:52.418709 20269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:52.420365 20269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:52.420488 20269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:52.420671 20269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:52.420939 20269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:52.420974 20269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:52.421000 20269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:52.421025 20269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:52.427310 20269 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:44691
I20260808 02:12:52.427328 20474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:44691 every 8 connection(s)
I20260808 02:12:52.427618 20269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:12:52.428920 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 20269
I20260808 02:12:52.429023 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:12:52.429190 19691 external_mini_cluster.cc:1428] Setting key 9e8b4708d37d405f09f919c5a032f896
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[19701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155172, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:52.436544 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.428426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:42061 (local address 127.19.58.254:42265)
0808 02:12:52.428498 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.428500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.428674 (+ 174us) server_negotiation.cc:407] Connection header received
0808 02:12:52.429329 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.429330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.429341 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.429396 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:52.430340 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.430851 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.431779 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.431973 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.432022 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.433999 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:52.434000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:52.434001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:52.434008 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:52.435246 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.435623 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.435624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.435624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.435678 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.435951 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.435952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.435952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.436063 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:52.436119 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.436364 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.436452 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:12:52.436687 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.428395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42265 (local address 127.19.58.195:42061)
0808 02:12:52.428557 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:52.428591 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:52.429194 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:52.429464 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:52.429469 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:52.430066 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:52.430263 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.430270 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.430918 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.430919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.431682 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:52.431684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.432032 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.432033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:52.432132 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.432555 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:52.432563 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:52.433910 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:52.435334 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:52.435340 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:52.435343 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:52.435556 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:52.435750 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:52.435751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:52.435751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:52.435891 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:52.436120 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:52.436126 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:52.436253 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 02:12:52.436338 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:12:52.436484 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260808 02:12:52.437084 20475 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42265
I20260808 02:12:52.437171 20475 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:52.437340 20475 heartbeater.cc:511] Master 127.19.58.254:42265 requested a full tablet report, sending...
I20260808 02:12:52.437757 19768 ts_manager.cc:194] Registered new tserver with Master: dbfe43701f4e4f609fa22f1065abb402 (127.19.58.195:44691)
I20260808 02:12:52.438130 19768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:42061
I20260808 02:12:52.441095 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[19701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155172, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[19701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155172, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:12:52.463132 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.456595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39310 (local address 127.19.58.193:40477)
0808 02:12:52.456757 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.456761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.456771 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:12:52.456795 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.456797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.456810 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.456869 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:52.457311 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.458025 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.458687 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.459008 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.459059 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.461020 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:52.461028 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:52.461028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:52.461034 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:52.462283 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.462566 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.462568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.462568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.462617 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.462733 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.462733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.462733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.462840 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:52.462965 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.462998 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.463059 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
W20260808 02:12:52.463475 19933 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39310
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[19701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[19701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:12:52.484802 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:52.478864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39324 (local address 127.19.58.193:40477)
0808 02:12:52.478938 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:52.478940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:52.478990 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:12:52.479033 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:52.479034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:52.479045 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:52.479089 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:52.479548 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.480080 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.480728 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:52.480919 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:52.480962 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:52.482810 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:52.482812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:52.482812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:52.482818 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:52.484053 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.484297 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.484297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.484298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.484342 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:52.484457 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:52.484457 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:52.484458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:52.484547 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:52.484630 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:52.484677 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 02:12:52.484730 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 02:12:52.485284 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 19861
I20260808 02:12:52.491403 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 20070
I20260808 02:12:52.504959 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 20269
I20260808 02:12:52.510838 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 19718
2026-08-08T02:12:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1976 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[20493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[20493](info): set up 2 sockets
Aug 08 02:12:52 dist-test-slave-1l3l krb5kdc[20493](info): commencing operation
krb5kdc: starting...
W20260808 02:12:53.621652 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:12:53 dist-test-slave-1l3l krb5kdc[20493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:12:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:12:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:12:53.743443 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:37195
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:37195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:53.822315 20515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:53.822497 20515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:53.822517 20515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:53.823619 20515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:12:53.823659 20515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:53.823673 20515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:53.823683 20515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:12:53.823696 20515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:12:53.825151 20515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:37195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:37195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:53.825369 20515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:53.825564 20515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:53.830188 20523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:53.830178 20522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:53.830156 20525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:53.830371 20515 server_base.cc:1034] running on GCE node
I20260808 02:12:53.830596 20515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:53.830849 20515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:53.832001 20515 hybrid_clock.cc:648] HybridClock initialized: now 1786155173831979 us; error 36 us; skew 500 ppm
Aug 08 02:12:53 dist-test-slave-1l3l krb5kdc[20493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155173, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:53.834335 20515 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:12:53.834735 20515 webserver.cc:460] Webserver started at http://127.19.58.254:35865/ using document root <none> and password file <none>
I20260808 02:12:53.834931 20515 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:53.834982 20515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:53.835107 20515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:53.836169 20515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "361edf74c05b4c4bbafa7005c364da5b"
format_stamp: "Formatted at 2026-08-08 02:12:53 on dist-test-slave-1l3l"
server_key: "e3f6648f8f531b9aab6c702bb33a1c20"
server_key_iv: "224f54196a2e5606a2dc2c337ddfd773"
server_key_version: "encryptionkey@0"
I20260808 02:12:53.836488 20515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "361edf74c05b4c4bbafa7005c364da5b"
format_stamp: "Formatted at 2026-08-08 02:12:53 on dist-test-slave-1l3l"
server_key: "e3f6648f8f531b9aab6c702bb33a1c20"
server_key_iv: "224f54196a2e5606a2dc2c337ddfd773"
server_key_version: "encryptionkey@0"
I20260808 02:12:53.837667 20515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:12:53.838313 20539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:53.838486 20515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:12:53.838558 20515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "361edf74c05b4c4bbafa7005c364da5b"
format_stamp: "Formatted at 2026-08-08 02:12:53 on dist-test-slave-1l3l"
server_key: "e3f6648f8f531b9aab6c702bb33a1c20"
server_key_iv: "224f54196a2e5606a2dc2c337ddfd773"
server_key_version: "encryptionkey@0"
I20260808 02:12:53.838608 20515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:53.848443 20515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:53.850376 20515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:53.850518 20515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:53.854210 20515 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:37195
I20260808 02:12:53.854262 20608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:37195 every 8 connection(s)
I20260808 02:12:53.854558 20515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:12:53.855404 20611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:53.857606 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 20515
I20260808 02:12:53.857697 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:12:53.857774 20611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b: Bootstrap starting.
I20260808 02:12:53.857867 19691 external_mini_cluster.cc:1428] Setting key c9dc4ea5a57931b081465a019910360a
I20260808 02:12:53.858428 20611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:53.858758 20611 log.cc:824] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:53.859504 20611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b: No bootstrap required, opened a new log
I20260808 02:12:53.861750 20611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "361edf74c05b4c4bbafa7005c364da5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37195 } }
I20260808 02:12:53.861881 20611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:53.861902 20611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 361edf74c05b4c4bbafa7005c364da5b, State: Initialized, Role: FOLLOWER
I20260808 02:12:53.862006 20611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "361edf74c05b4c4bbafa7005c364da5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37195 } }
I20260808 02:12:53.862076 20611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:12:53.862106 20611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:12:53.862150 20611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:53.862651 20611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "361edf74c05b4c4bbafa7005c364da5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37195 } }
I20260808 02:12:53.862746 20611 leader_election.cc:302] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 361edf74c05b4c4bbafa7005c364da5b; no voters:
I20260808 02:12:53.862919 20611 leader_election.cc:288] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:12:53.863013 20616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:53.863168 20611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:12:53.863202 20616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [term 1 LEADER]: Becoming Leader. State: Replica: 361edf74c05b4c4bbafa7005c364da5b, State: Running, Role: LEADER
I20260808 02:12:53.863386 20616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "361edf74c05b4c4bbafa7005c364da5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37195 } }
I20260808 02:12:53.863770 20619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 361edf74c05b4c4bbafa7005c364da5b. Latest consensus state: current_term: 1 leader_uuid: "361edf74c05b4c4bbafa7005c364da5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "361edf74c05b4c4bbafa7005c364da5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37195 } } }
I20260808 02:12:53.863852 20619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:53.864178 20618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "361edf74c05b4c4bbafa7005c364da5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "361edf74c05b4c4bbafa7005c364da5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37195 } } }
I20260808 02:12:53.864252 20618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b [sys.catalog]: This master's current role is: LEADER
Aug 08 02:12:53 dist-test-slave-1l3l krb5kdc[20493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
W20260808 02:12:53.865523 20640 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:12:53.865621 20640 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:12:53.865685 20628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:12:53.865820 20628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:12:53.867156 20628 catalog_manager.cc:1347] Generated new cluster ID: f9d239196ead4812a1d342bbc6be4e95
I20260808 02:12:53.867209 20628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:12:53.868381 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:53.858355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.19.58.254:37195)
0808 02:12:53.858577 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:53.858588 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:53.858622 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:12:53.859142 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:53.859149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:53.859205 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:53.859344 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:53.859943 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:53.861290 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:53.862031 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:53.862504 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:53.862602 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:53.865159 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:53.865164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:53.865198 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:53.865227 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:53.866916 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:53.867285 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:53.867288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:53.867288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:53.867375 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:53.867556 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:53.867557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:53.867557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:53.867826 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:53.867945 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:53.868064 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 02:12:53.868190 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":60,"threads_started":1}
I20260808 02:12:53.874732 20628 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:12:53.875218 20628 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:12:53.880934 20628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 361edf74c05b4c4bbafa7005c364da5b: Generated new TSK 0
I20260808 02:12:53.881078 20628 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:12:53.908603 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:53.987421 20649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:53.987622 20649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:53.987643 20649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:53.988771 20649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:53.988813 20649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:53.988838 20649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:12:53.990286 20649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37195
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:53.990497 20649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:53.990716 20649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:53.995545 20654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:53.995550 20655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:53.995584 20649 server_base.cc:1034] running on GCE node
W20260808 02:12:53.995568 20659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:53.995973 20649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:53.996174 20649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:53.997310 20649 hybrid_clock.cc:648] HybridClock initialized: now 1786155173997297 us; error 33 us; skew 500 ppm
Aug 08 02:12:53 dist-test-slave-1l3l krb5kdc[20493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155173, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:53.999809 20649 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:12:54.000252 20649 webserver.cc:460] Webserver started at http://127.19.58.193:40311/ using document root <none> and password file <none>
I20260808 02:12:54.000536 20649 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:54.000598 20649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:54.000725 20649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:54.001758 20649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "28dec1ae54c640f980855dd88b2b939c"
format_stamp: "Formatted at 2026-08-08 02:12:53 on dist-test-slave-1l3l"
server_key: "c6a241269c6d37d709ec8951b7a85247"
server_key_iv: "2a072fbaeed783010ca9b627157115bd"
server_key_version: "encryptionkey@0"
I20260808 02:12:54.002099 20649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "28dec1ae54c640f980855dd88b2b939c"
format_stamp: "Formatted at 2026-08-08 02:12:53 on dist-test-slave-1l3l"
server_key: "c6a241269c6d37d709ec8951b7a85247"
server_key_iv: "2a072fbaeed783010ca9b627157115bd"
server_key_version: "encryptionkey@0"
I20260808 02:12:54.003537 20649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:12:54.004287 20670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.004452 20649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.004536 20649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "28dec1ae54c640f980855dd88b2b939c"
format_stamp: "Formatted at 2026-08-08 02:12:53 on dist-test-slave-1l3l"
server_key: "c6a241269c6d37d709ec8951b7a85247"
server_key_iv: "2a072fbaeed783010ca9b627157115bd"
server_key_version: "encryptionkey@0"
I20260808 02:12:54.004606 20649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:54.022904 20649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:54.024705 20649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:54.024839 20649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:54.025038 20649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:54.025321 20649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:54.025355 20649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.025383 20649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:54.025406 20649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.031133 20649 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:36335
I20260808 02:12:54.031172 20840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:36335 every 8 connection(s)
I20260808 02:12:54.031466 20649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:12:54.032842 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 20649
I20260808 02:12:54.032917 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:12:54.033056 19691 external_mini_cluster.cc:1428] Setting key ec886b0cb6471dfd23c6a37b9d82786d
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[20493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155173, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:54.040365 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.032149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:58749 (local address 127.19.58.254:37195)
0808 02:12:54.032222 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.032224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.032385 (+ 161us) server_negotiation.cc:407] Connection header received
0808 02:12:54.033399 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.033400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.033411 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.033467 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:54.034038 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.034537 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.035532 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.035802 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.035859 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.037811 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:54.037813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:54.037813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:54.037820 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:54.039107 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.039458 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.039459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.039460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.039519 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.039742 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.039743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.039743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.039857 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:54.039914 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.040169 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.040272 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:12:54.040501 20844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.032119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:37195 (local address 127.19.58.193:58749)
0808 02:12:54.032277 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:54.032312 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:54.033325 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:54.033520 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:54.033524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:54.033758 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:54.033947 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.033951 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.034618 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.034619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.035455 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.035458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.035868 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.035870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.035992 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.036440 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:54.036447 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:54.037724 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:54.039178 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:54.039185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:54.039187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:54.039390 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:54.039572 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:54.039572 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:54.039573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:54.039673 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:54.039917 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:54.039924 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:54.040052 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 02:12:54.040137 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:12:54.040296 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260808 02:12:54.040922 20841 heartbeater.cc:347] Connected to a master server at 127.19.58.254:37195
I20260808 02:12:54.041011 20841 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
I20260808 02:12:54.041186 20841 heartbeater.cc:511] Master 127.19.58.254:37195 requested a full tablet report, sending...
I20260808 02:12:54.041672 20558 ts_manager.cc:194] Registered new tserver with Master: 28dec1ae54c640f980855dd88b2b939c (127.19.58.193:36335)
I20260808 02:12:54.042474 20558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:58749
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:12:54.056826 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:54.135982 20848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:54.136173 20848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:54.136194 20848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:54.137302 20848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:54.137341 20848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:54.137365 20848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:12:54.138798 20848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37195
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:54.139016 20848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:54.139213 20848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:54.143983 20859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:54.144003 20861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:54.143990 20866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:54.144172 20848 server_base.cc:1034] running on GCE node
I20260808 02:12:54.144418 20848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:54.144635 20848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:54.145764 20848 hybrid_clock.cc:648] HybridClock initialized: now 1786155174145755 us; error 38 us; skew 500 ppm
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[20493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155174, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:54.148217 20848 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:12:54.148631 20848 webserver.cc:460] Webserver started at http://127.19.58.194:34461/ using document root <none> and password file <none>
I20260808 02:12:54.148844 20848 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:54.148895 20848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:54.149019 20848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:54.149995 20848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "229866574a9b48d49ebf44d7577233bc"
format_stamp: "Formatted at 2026-08-08 02:12:54 on dist-test-slave-1l3l"
server_key: "99f14e412fc2b956f8268130360edf22"
server_key_iv: "e74bb5cf31c3633f9e600b307945952d"
server_key_version: "encryptionkey@0"
I20260808 02:12:54.150317 20848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "229866574a9b48d49ebf44d7577233bc"
format_stamp: "Formatted at 2026-08-08 02:12:54 on dist-test-slave-1l3l"
server_key: "99f14e412fc2b956f8268130360edf22"
server_key_iv: "e74bb5cf31c3633f9e600b307945952d"
server_key_version: "encryptionkey@0"
I20260808 02:12:54.151582 20848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 02:12:54.152272 20873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.152413 20848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.152486 20848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "229866574a9b48d49ebf44d7577233bc"
format_stamp: "Formatted at 2026-08-08 02:12:54 on dist-test-slave-1l3l"
server_key: "99f14e412fc2b956f8268130360edf22"
server_key_iv: "e74bb5cf31c3633f9e600b307945952d"
server_key_version: "encryptionkey@0"
I20260808 02:12:54.152549 20848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:54.163548 20848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:54.165580 20848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:54.165701 20848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:54.165903 20848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:54.166184 20848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:54.166217 20848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.166245 20848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:54.166270 20848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.172010 20848 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:45417
I20260808 02:12:54.172076 21057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:45417 every 8 connection(s)
I20260808 02:12:54.172356 20848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[20493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155174, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:54.180613 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.172999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:53941 (local address 127.19.58.254:37195)
0808 02:12:54.173081 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.173084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.173233 (+ 149us) server_negotiation.cc:407] Connection header received
0808 02:12:54.174098 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.174099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.174109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.174161 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:54.174705 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.175205 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.176241 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.176439 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.176489 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.178438 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:54.178440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:54.178440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:54.178447 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:54.179556 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.179937 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.179937 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.179938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.179991 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.180162 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.180162 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.180163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.180255 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:54.180298 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.180487 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.180549 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 02:12:54.180779 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.172964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:37195 (local address 127.19.58.194:53941)
0808 02:12:54.173116 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:54.173152 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:54.174016 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:54.174214 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:54.174217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:54.174461 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:54.174641 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.174646 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.175280 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.175281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.176176 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.176179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.176497 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.176497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.176589 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.176988 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:54.176996 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:54.178329 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:54.179623 (+ 1294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:54.179629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:54.179630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:54.179879 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:54.180039 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:54.180040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:54.180040 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:54.180127 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:54.180294 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:54.180296 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:54.180402 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 02:12:54.180476 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:12:54.180586 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260808 02:12:54.181180 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 20848
I20260808 02:12:54.181265 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:12:54.181273 21058 heartbeater.cc:347] Connected to a master server at 127.19.58.254:37195
I20260808 02:12:54.181373 21058 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:54.181428 19691 external_mini_cluster.cc:1428] Setting key b3db646b05e8937cd20cab1a1c24f508
I20260808 02:12:54.181553 21058 heartbeater.cc:511] Master 127.19.58.254:37195 requested a full tablet report, sending...
I20260808 02:12:54.181893 20557 ts_manager.cc:194] Registered new tserver with Master: 229866574a9b48d49ebf44d7577233bc (127.19.58.194:45417)
I20260808 02:12:54.182258 20557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:53941
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:12:54.204097 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:54.283181 21067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:54.283375 21067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:54.283397 21067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:54.284562 21067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:54.284602 21067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:54.284627 21067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:12:54.286054 21067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37195
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:54.286259 21067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:54.286469 21067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:54.291117 21078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:54.291142 21074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:54.291167 21075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:54.291263 21067 server_base.cc:1034] running on GCE node
I20260808 02:12:54.291487 21067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:54.291671 21067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:54.292827 21067 hybrid_clock.cc:648] HybridClock initialized: now 1786155174292814 us; error 29 us; skew 500 ppm
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[20493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155174, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:54.295275 21067 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:12:54.295763 21067 webserver.cc:460] Webserver started at http://127.19.58.195:39197/ using document root <none> and password file <none>
I20260808 02:12:54.295970 21067 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:54.296018 21067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:54.296141 21067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:54.297116 21067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "2c05b940b3d342f28eb6741345c6536c"
format_stamp: "Formatted at 2026-08-08 02:12:54 on dist-test-slave-1l3l"
server_key: "98a8e2102038adbc87b970b4318fec5a"
server_key_iv: "8d232287bdba9e4dee029bf5ae717d9b"
server_key_version: "encryptionkey@0"
I20260808 02:12:54.297525 21067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "2c05b940b3d342f28eb6741345c6536c"
format_stamp: "Formatted at 2026-08-08 02:12:54 on dist-test-slave-1l3l"
server_key: "98a8e2102038adbc87b970b4318fec5a"
server_key_iv: "8d232287bdba9e4dee029bf5ae717d9b"
server_key_version: "encryptionkey@0"
I20260808 02:12:54.299193 21067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:12:54.300030 21089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.300297 21067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:54.300384 21067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "2c05b940b3d342f28eb6741345c6536c"
format_stamp: "Formatted at 2026-08-08 02:12:54 on dist-test-slave-1l3l"
server_key: "98a8e2102038adbc87b970b4318fec5a"
server_key_iv: "8d232287bdba9e4dee029bf5ae717d9b"
server_key_version: "encryptionkey@0"
I20260808 02:12:54.300454 21067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:54.324115 21067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:54.326258 21067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:54.326402 21067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:54.326622 21067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:12:54.326923 21067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:54.326957 21067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.326987 21067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:54.327009 21067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:54.333051 21067 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:41069
I20260808 02:12:54.333112 21260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:41069 every 8 connection(s)
I20260808 02:12:54.333393 21067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:12:54.338655 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 21067
I20260808 02:12:54.338739 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:12:54.338910 19691 external_mini_cluster.cc:1428] Setting key b282c83a0a128796ad935a9e1ba5c670
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[20493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155174, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:54.342510 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.334044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:56647 (local address 127.19.58.254:37195)
0808 02:12:54.334089 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.334092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.334274 (+ 182us) server_negotiation.cc:407] Connection header received
0808 02:12:54.335190 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.335191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.335203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.335253 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:54.335910 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.336419 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.337425 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.337629 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.337685 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.339854 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:54.339855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:54.339855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:54.339861 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:54.341278 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.341743 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.341744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.341744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.341807 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.342051 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.342052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.342052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.342150 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:54.342197 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.342393 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.342449 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:12:54.342737 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.334002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:37195 (local address 127.19.58.195:56647)
0808 02:12:54.334152 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:54.334191 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:54.335084 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:54.335346 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:54.335350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:54.335598 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:54.335827 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.335832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.336508 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.336509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.337340 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.337342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.337697 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.337698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.337824 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.338220 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:54.338227 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:54.339791 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:54.341376 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:54.341383 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:54.341385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:54.341674 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:54.341867 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:54.341868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:54.341869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:54.341995 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:54.342189 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:54.342191 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:54.342296 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 02:12:54.342371 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:12:54.342488 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260808 02:12:54.343123 21264 heartbeater.cc:347] Connected to a master server at 127.19.58.254:37195
I20260808 02:12:54.343201 21264 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:54.343334 21264 heartbeater.cc:511] Master 127.19.58.254:37195 requested a full tablet report, sending...
I20260808 02:12:54.343631 20557 ts_manager.cc:194] Registered new tserver with Master: 2c05b940b3d342f28eb6741345c6536c (127.19.58.195:41069)
I20260808 02:12:54.344056 20557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:56647
I20260808 02:12:54.350528 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:12:54.357151 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.351985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.19.58.254:37195)
0808 02:12:54.352044 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.352046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.352140 (+ 94us) server_negotiation.cc:407] Connection header received
0808 02:12:54.352177 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.352178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.352186 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.352224 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:54.352728 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.353171 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.353818 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.353996 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.354040 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.355072 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:54.355073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:54.355073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:54.355079 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:54.356339 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.356605 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.356605 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.356605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.356647 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.356761 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.356762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.356762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.356873 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:54.356953 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.357023 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.357079 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:12:54.359220 20557 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:54.360745 20557 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:54.366633 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.362620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:45417 (local address 127.0.0.1:51162)
0808 02:12:54.362910 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:54.362920 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:54.362990 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:54.363222 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:54.363223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:54.363225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:54.363261 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.363262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.364707 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.364708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.365623 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.365625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.366327 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.366327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.366425 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.366444 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 02:12:54.366508 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 02:12:54.366560 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":54,"threads_started":1}
I20260808 02:12:54.366643 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.362740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51162 (local address 127.19.58.194:45417)
0808 02:12:54.362904 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.362909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.362981 (+ 72us) server_negotiation.cc:407] Connection header received
0808 02:12:54.363050 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.363052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.363066 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.363135 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:54.363332 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.364620 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.365698 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.366249 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.366318 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.366368 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.366527 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.366571 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260808 02:12:54.367060 21289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.362793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:36335 (local address 127.0.0.1:54250)
0808 02:12:54.363173 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:54.363206 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:54.363255 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:54.363403 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:54.363404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:54.363405 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:54.363455 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.363456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.364604 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.364605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.365968 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.365971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.366847 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.366848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.366909 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.366913 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:12:54.366955 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 02:12:54.366993 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":26,"thread_start_us":30,"threads_started":1}
I20260808 02:12:54.367100 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.362977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54250 (local address 127.19.58.193:36335)
0808 02:12:54.363186 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.363190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.363248 (+ 58us) server_negotiation.cc:407] Connection header received
0808 02:12:54.363315 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.363318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.363332 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.363400 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:54.363620 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.364493 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.366058 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.366688 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.366757 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.366814 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.366994 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.367039 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260808 02:12:54.367400 20958 tablet_service.cc:1467] Processing CreateTablet for tablet 480d0c9cc1d74488b8996886fbb6f680 (DEFAULT_TABLE table=test-table [id=083016c00d804f53beda6ba194860899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:54.367779 20958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 480d0c9cc1d74488b8996886fbb6f680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:54.367672 20747 tablet_service.cc:1467] Processing CreateTablet for tablet 480d0c9cc1d74488b8996886fbb6f680 (DEFAULT_TABLE table=test-table [id=083016c00d804f53beda6ba194860899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:54.368029 20747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 480d0c9cc1d74488b8996886fbb6f680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:54.368213 21282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.362608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:41069 (local address 127.0.0.1:39044)
0808 02:12:54.362762 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:54.362797 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:54.362860 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:54.363162 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:54.363165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:54.363167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:54.363225 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.363227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.364559 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.364560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.365968 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.365971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.367985 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.367986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.368048 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.368053 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:12:54.368111 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 02:12:54.368150 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":57,"thread_start_us":58,"threads_started":1}
I20260808 02:12:54.368471 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.362670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39044 (local address 127.19.58.195:41069)
0808 02:12:54.362995 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.362998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.363006 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:12:54.363042 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.363044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.363059 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.363299 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:54.363370 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.364459 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.367200 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.367897 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.367962 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.368012 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.368130 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.368170 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":75,"threads_started":1}
I20260808 02:12:54.370433 21291 tablet_bootstrap.cc:492] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc: Bootstrap starting.
I20260808 02:12:54.370818 21292 tablet_bootstrap.cc:492] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c: Bootstrap starting.
I20260808 02:12:54.371022 21169 tablet_service.cc:1467] Processing CreateTablet for tablet 480d0c9cc1d74488b8996886fbb6f680 (DEFAULT_TABLE table=test-table [id=083016c00d804f53beda6ba194860899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:54.371127 21291 tablet_bootstrap.cc:654] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:54.371295 21169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 480d0c9cc1d74488b8996886fbb6f680. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:54.371456 21292 tablet_bootstrap.cc:654] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:54.371701 21291 log.cc:824] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:54.372037 21292 log.cc:824] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:54.372519 21291 tablet_bootstrap.cc:492] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc: No bootstrap required, opened a new log
I20260808 02:12:54.372598 21291 ts_tablet_manager.cc:1397] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:54.372677 21292 tablet_bootstrap.cc:492] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c: No bootstrap required, opened a new log
I20260808 02:12:54.372743 21292 ts_tablet_manager.cc:1397] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:54.374106 21292 raft_consensus.cc:348] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c05b940b3d342f28eb6741345c6536c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41069 } } peers { permanent_uuid: "229866574a9b48d49ebf44d7577233bc" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 45417 } } peers { permanent_uuid: "28dec1ae54c640f980855dd88b2b939c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 36335 } }
I20260808 02:12:54.374240 21292 raft_consensus.cc:374] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:54.374279 21292 raft_consensus.cc:729] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28dec1ae54c640f980855dd88b2b939c, State: Initialized, Role: FOLLOWER
I20260808 02:12:54.374457 21299 tablet_bootstrap.cc:492] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c: Bootstrap starting.
I20260808 02:12:54.374460 21291 raft_consensus.cc:348] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c05b940b3d342f28eb6741345c6536c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41069 } } peers { permanent_uuid: "229866574a9b48d49ebf44d7577233bc" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 45417 } } peers { permanent_uuid: "28dec1ae54c640f980855dd88b2b939c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 36335 } }
I20260808 02:12:54.374605 21291 raft_consensus.cc:374] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:54.374586 21292 consensus_queue.cc:260] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c05b940b3d342f28eb6741345c6536c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41069 } } peers { permanent_uuid: "229866574a9b48d49ebf44d7577233bc" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 45417 } } peers { permanent_uuid: "28dec1ae54c640f980855dd88b2b939c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 36335 } }
I20260808 02:12:54.374645 21291 raft_consensus.cc:729] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229866574a9b48d49ebf44d7577233bc, State: Initialized, Role: FOLLOWER
I20260808 02:12:54.374744 21291 consensus_queue.cc:260] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c05b940b3d342f28eb6741345c6536c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41069 } } peers { permanent_uuid: "229866574a9b48d49ebf44d7577233bc" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 45417 } } peers { permanent_uuid: "28dec1ae54c640f980855dd88b2b939c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 36335 } }
I20260808 02:12:54.374879 21292 ts_tablet_manager.cc:1428] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:54.374950 20841 heartbeater.cc:503] Master 127.19.58.254:37195 was elected leader, sending a full tablet report...
I20260808 02:12:54.375011 21291 ts_tablet_manager.cc:1428] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:12:54.375077 21058 heartbeater.cc:503] Master 127.19.58.254:37195 was elected leader, sending a full tablet report...
I20260808 02:12:54.375095 21299 tablet_bootstrap.cc:654] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:54.375681 21299 log.cc:824] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:54.376353 21299 tablet_bootstrap.cc:492] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c: No bootstrap required, opened a new log
I20260808 02:12:54.376415 21299 ts_tablet_manager.cc:1397] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:54.377674 21299 raft_consensus.cc:348] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c05b940b3d342f28eb6741345c6536c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41069 } } peers { permanent_uuid: "229866574a9b48d49ebf44d7577233bc" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 45417 } } peers { permanent_uuid: "28dec1ae54c640f980855dd88b2b939c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 36335 } }
I20260808 02:12:54.377797 21299 raft_consensus.cc:374] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:54.377828 21299 raft_consensus.cc:729] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c05b940b3d342f28eb6741345c6536c, State: Initialized, Role: FOLLOWER
I20260808 02:12:54.377930 21299 consensus_queue.cc:260] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c05b940b3d342f28eb6741345c6536c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41069 } } peers { permanent_uuid: "229866574a9b48d49ebf44d7577233bc" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 45417 } } peers { permanent_uuid: "28dec1ae54c640f980855dd88b2b939c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 36335 } }
I20260808 02:12:54.378155 21299 ts_tablet_manager.cc:1428] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:54.378176 21264 heartbeater.cc:503] Master 127.19.58.254:37195 was elected leader, sending a full tablet report...
W20260808 02:12:54.422883 21060 tablet.cc:2369] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:12:54.532236 20842 tablet.cc:2369] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:12:54.583959 21265 tablet.cc:2369] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:12:54.635741 21306 raft_consensus.cc:482] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:54.635872 21306 raft_consensus.cc:504] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c05b940b3d342f28eb6741345c6536c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41069 } } peers { permanent_uuid: "229866574a9b48d49ebf44d7577233bc" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 45417 } } peers { permanent_uuid: "28dec1ae54c640f980855dd88b2b939c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 36335 } }
I20260808 02:12:54.636186 21306 leader_election.cc:288] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 229866574a9b48d49ebf44d7577233bc (127.19.58.194:45417), 28dec1ae54c640f980855dd88b2b939c (127.19.58.193:36335)
I20260808 02:12:54.638934 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.636480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:41433 (local address 127.19.58.194:45417)
0808 02:12:54.636565 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.636567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.636574 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:12:54.636606 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.636607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.636617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.636664 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:54.636812 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.637307 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.638235 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.638615 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.638679 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.638687 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.638827 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.638864 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 02:12:54.638934 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.636385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:45417 (local address 127.19.58.195:41433)
0808 02:12:54.636443 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:54.636457 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:54.636522 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:54.636680 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:54.636681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:54.636682 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:54.636757 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.636759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.637383 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.637384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.638147 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.638150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.638688 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.638689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.638769 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.638771 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:12:54.638814 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 02:12:54.638855 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":1}
I20260808 02:12:54.639004 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.636569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:59335 (local address 127.19.58.193:36335)
0808 02:12:54.636617 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.636619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.636765 (+ 146us) server_negotiation.cc:407] Connection header received
0808 02:12:54.636818 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.636819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.636828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.636893 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:54.637032 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.637487 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.638364 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.638706 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.638765 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.638773 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.638906 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.638942 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:12:54.639209 21307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.636497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:36335 (local address 127.19.58.195:59335)
0808 02:12:54.636689 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:54.636698 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:54.636758 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:54.636886 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:54.636886 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:54.636888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:54.636971 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.636972 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.637565 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.637565 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.638280 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:54.638281 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.638778 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.638779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:54.638841 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.638844 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:12:54.638890 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:12:54.638926 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":126,"threads_started":1}
I20260808 02:12:54.639376 20984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "480d0c9cc1d74488b8996886fbb6f680" candidate_uuid: "2c05b940b3d342f28eb6741345c6536c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229866574a9b48d49ebf44d7577233bc" is_pre_election: true
I20260808 02:12:54.639496 20984 raft_consensus.cc:2454] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c05b940b3d342f28eb6741345c6536c in term 0.
I20260808 02:12:54.639673 21092 leader_election.cc:302] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229866574a9b48d49ebf44d7577233bc, 2c05b940b3d342f28eb6741345c6536c; no voters:
I20260808 02:12:54.639808 21306 raft_consensus.cc:2781] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:54.639864 21306 raft_consensus.cc:482] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:54.639883 21306 raft_consensus.cc:3037] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:54.639824 20775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "480d0c9cc1d74488b8996886fbb6f680" candidate_uuid: "2c05b940b3d342f28eb6741345c6536c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28dec1ae54c640f980855dd88b2b939c" is_pre_election: true
I20260808 02:12:54.640002 20775 raft_consensus.cc:2454] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c05b940b3d342f28eb6741345c6536c in term 0.
I20260808 02:12:54.640707 21306 raft_consensus.cc:504] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c05b940b3d342f28eb6741345c6536c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41069 } } peers { permanent_uuid: "229866574a9b48d49ebf44d7577233bc" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 45417 } } peers { permanent_uuid: "28dec1ae54c640f980855dd88b2b939c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 36335 } }
I20260808 02:12:54.640811 21306 leader_election.cc:288] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [CANDIDATE]: Term 1 election: Requested vote from peers 229866574a9b48d49ebf44d7577233bc (127.19.58.194:45417), 28dec1ae54c640f980855dd88b2b939c (127.19.58.193:36335)
I20260808 02:12:54.640945 20984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "480d0c9cc1d74488b8996886fbb6f680" candidate_uuid: "2c05b940b3d342f28eb6741345c6536c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229866574a9b48d49ebf44d7577233bc"
I20260808 02:12:54.641011 20984 raft_consensus.cc:3037] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:54.641000 20775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "480d0c9cc1d74488b8996886fbb6f680" candidate_uuid: "2c05b940b3d342f28eb6741345c6536c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28dec1ae54c640f980855dd88b2b939c"
I20260808 02:12:54.641075 20775 raft_consensus.cc:3037] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:54.641737 20984 raft_consensus.cc:2454] T 480d0c9cc1d74488b8996886fbb6f680 P 229866574a9b48d49ebf44d7577233bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c05b940b3d342f28eb6741345c6536c in term 1.
I20260808 02:12:54.641880 21092 leader_election.cc:302] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229866574a9b48d49ebf44d7577233bc, 2c05b940b3d342f28eb6741345c6536c; no voters:
I20260808 02:12:54.641880 20775 raft_consensus.cc:2454] T 480d0c9cc1d74488b8996886fbb6f680 P 28dec1ae54c640f980855dd88b2b939c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c05b940b3d342f28eb6741345c6536c in term 1.
I20260808 02:12:54.641957 21306 raft_consensus.cc:2781] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:54.642107 21306 raft_consensus.cc:686] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [term 1 LEADER]: Becoming Leader. State: Replica: 2c05b940b3d342f28eb6741345c6536c, State: Running, Role: LEADER
I20260808 02:12:54.642241 21306 consensus_queue.cc:237] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c05b940b3d342f28eb6741345c6536c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41069 } } peers { permanent_uuid: "229866574a9b48d49ebf44d7577233bc" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 45417 } } peers { permanent_uuid: "28dec1ae54c640f980855dd88b2b939c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 36335 } }
I20260808 02:12:54.643026 20557 catalog_manager.cc:5495] T 480d0c9cc1d74488b8996886fbb6f680 P 2c05b940b3d342f28eb6741345c6536c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c05b940b3d342f28eb6741345c6536c (127.19.58.195). New cstate: current_term: 1 leader_uuid: "2c05b940b3d342f28eb6741345c6536c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c05b940b3d342f28eb6741345c6536c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "229866574a9b48d49ebf44d7577233bc" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 45417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28dec1ae54c640f980855dd88b2b939c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 36335 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[20493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:12:54.651003 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.644614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.19.58.193:36335)
0808 02:12:54.644652 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.644654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.644794 (+ 140us) server_negotiation.cc:407] Connection header received
0808 02:12:54.644832 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.644832 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.644840 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.644881 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:54.645338 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.645752 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.646352 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.646521 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.646561 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.648725 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:54.648734 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:54.648734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:54.648742 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:54.649987 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.650293 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.650296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.650296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.650354 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.650522 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.650523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.650523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.650645 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:54.650770 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.650836 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.650923 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[20493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155174, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[20493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155174, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:12:54.673708 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.667350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54282 (local address 127.19.58.193:36335)
0808 02:12:54.667404 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.667406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.667474 (+ 68us) server_negotiation.cc:407] Connection header received
0808 02:12:54.667532 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.667533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.667543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.667587 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:54.668148 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.668608 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.669279 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.669461 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.669515 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.671487 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:54.671488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:54.671488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:54.671494 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:54.672635 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.672997 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.672998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.672998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.673057 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.673259 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.673259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.673259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.673376 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:54.673448 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.673580 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.673646 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[20493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[20493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:12:54.694802 21290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:54.688675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54298 (local address 127.19.58.193:36335)
0808 02:12:54.688725 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:54.688727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:54.688816 (+ 89us) server_negotiation.cc:407] Connection header received
0808 02:12:54.688880 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:54.688881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:54.688890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:54.688936 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:54.689455 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.689935 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.690627 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:54.690806 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:54.690859 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:54.692828 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:54.692830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:54.692830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:54.692837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:54.693850 (+ 1013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.694135 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.694136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.694137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.694185 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:54.694343 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:54.694345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:54.694345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:54.694451 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:54.694536 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:54.694644 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 02:12:54.694714 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 02:12:54.695221 20711 tablet_service.cc:2908] Rejecting scan request for tablet 480d0c9cc1d74488b8996886fbb6f680: Uninitialized: clean time has not yet been initialized
I20260808 02:12:54.696012 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 20649
I20260808 02:12:54.713972 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 20848
I20260808 02:12:54.722868 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 21067
I20260808 02:12:54.735553 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 20515
2026-08-08T02:12:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2226 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[21327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[21327](info): set up 2 sockets
Aug 08 02:12:54 dist-test-slave-1l3l krb5kdc[21327](info): commencing operation
krb5kdc: starting...
W20260808 02:12:55.845752 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:12:55 dist-test-slave-1l3l krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:12:55.956635 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40063
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:33913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:40063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:56.035455 21348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:56.035656 21348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:56.035686 21348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:56.036834 21348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:12:56.036875 21348 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:12:56.036892 21348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:56.036911 21348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:56.036926 21348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:12:56.036940 21348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:12:56.038380 21348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:33913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:40063
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:56.038611 21348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:56.038831 21348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:56.043519 21357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:56.043509 21355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:56.043641 21348 server_base.cc:1034] running on GCE node
W20260808 02:12:56.043493 21354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:56.043938 21348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:56.044194 21348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:56.045341 21348 hybrid_clock.cc:648] HybridClock initialized: now 1786155176045329 us; error 33 us; skew 500 ppm
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:56.052027 21348 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:12:56.052465 21348 webserver.cc:460] Webserver started at http://127.19.58.254:46735/ using document root <none> and password file <none>
I20260808 02:12:56.052666 21348 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:56.052714 21348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:56.052839 21348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:56.053886 21348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "a9e273501c054c439fbe8da2b4bb98e2"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "86d476f605cb87225c3e9c9284952c3c"
server_key_iv: "f7bc38c1cce78b829fb540ccf234b361"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.054229 21348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "a9e273501c054c439fbe8da2b4bb98e2"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "86d476f605cb87225c3e9c9284952c3c"
server_key_iv: "f7bc38c1cce78b829fb540ccf234b361"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.055598 21348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 02:12:56.056371 21365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.056545 21348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:12:56.056645 21348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "a9e273501c054c439fbe8da2b4bb98e2"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "86d476f605cb87225c3e9c9284952c3c"
server_key_iv: "f7bc38c1cce78b829fb540ccf234b361"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.056710 21348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:56.079466 21348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:56.081647 21348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:56.081789 21348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:56.086406 21348 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:40063
I20260808 02:12:56.086475 21453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:40063 every 8 connection(s)
I20260808 02:12:56.086758 21348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:12:56.087445 21456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:56.089756 21456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2: Bootstrap starting.
I20260808 02:12:56.090332 21456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:56.090598 21456 log.cc:824] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:56.090806 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 21348
I20260808 02:12:56.090912 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:12:56.091051 19691 external_mini_cluster.cc:1428] Setting key acfe5cdc2fe1ad087614b6b8aebf0616
I20260808 02:12:56.091297 21456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2: No bootstrap required, opened a new log
I20260808 02:12:56.094022 21456 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e273501c054c439fbe8da2b4bb98e2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40063 } }
I20260808 02:12:56.094156 21456 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:56.094183 21456 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e273501c054c439fbe8da2b4bb98e2, State: Initialized, Role: FOLLOWER
I20260808 02:12:56.094292 21456 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e273501c054c439fbe8da2b4bb98e2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40063 } }
I20260808 02:12:56.094364 21456 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:12:56.094386 21456 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:12:56.094431 21456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:56.095105 21456 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e273501c054c439fbe8da2b4bb98e2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40063 } }
I20260808 02:12:56.095211 21456 leader_election.cc:302] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9e273501c054c439fbe8da2b4bb98e2; no voters:
I20260808 02:12:56.095372 21456 leader_election.cc:288] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:12:56.095449 21466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:56.095618 21456 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:12:56.095629 21466 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [term 1 LEADER]: Becoming Leader. State: Replica: a9e273501c054c439fbe8da2b4bb98e2, State: Running, Role: LEADER
I20260808 02:12:56.095832 21466 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e273501c054c439fbe8da2b4bb98e2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40063 } }
I20260808 02:12:56.096163 21467 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9e273501c054c439fbe8da2b4bb98e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e273501c054c439fbe8da2b4bb98e2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40063 } } }
I20260808 02:12:56.096179 21468 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9e273501c054c439fbe8da2b4bb98e2. Latest consensus state: current_term: 1 leader_uuid: "a9e273501c054c439fbe8da2b4bb98e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9e273501c054c439fbe8da2b4bb98e2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40063 } } }
I20260808 02:12:56.096256 21467 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:56.096259 21468 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:56.096575 21479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:12:56.097723 21479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:56.099351 21479 catalog_manager.cc:1347] Generated new cluster ID: beae1b36d7e9450f841c55fa1591324f
I20260808 02:12:56.099408 21479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:12:56.102664 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.091502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.19.58.254:40063)
0808 02:12:56.091676 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.091685 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.091740 (+ 55us) server_negotiation.cc:407] Connection header received
0808 02:12:56.092574 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.092581 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.092627 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.092735 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:56.093506 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.094560 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.095576 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.096204 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.096295 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.099234 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:56.099238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:56.099239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:56.099251 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:56.101128 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.101490 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.101493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.101494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.101584 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.101744 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.101744 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.101745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.102009 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:56.102132 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.102359 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.102485 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260808 02:12:56.115939 21479 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:12:56.116482 21479 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:12:56.127842 21479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2: Generated new TSK 0
I20260808 02:12:56.128033 21479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:12:56.153963 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40063
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:33913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:56.233341 21496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:56.233541 21496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:56.233564 21496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:56.234647 21496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:56.234685 21496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:56.234704 21496 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:56.234720 21496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:12:56.236115 21496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:33913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.58.254:40063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:56.236295 21496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:56.236474 21496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:56.241092 21506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:56.241101 21503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:56.241113 21504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:56.241219 21496 server_base.cc:1034] running on GCE node
I20260808 02:12:56.241518 21496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:56.241746 21496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:56.242882 21496 hybrid_clock.cc:648] HybridClock initialized: now 1786155176242869 us; error 39 us; skew 500 ppm
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:56.245229 21496 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:12:56.245579 21496 webserver.cc:460] Webserver started at http://127.19.58.193:42059/ using document root <none> and password file <none>
I20260808 02:12:56.245750 21496 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:56.245790 21496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:56.245877 21496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:56.246825 21496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "6ded34b5968349b4a524a0357370d29f"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "ebd66322437f5ba8894c07084c53a5ac"
server_key_iv: "98285e831445f8ab3463c3b58341b257"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.247139 21496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "6ded34b5968349b4a524a0357370d29f"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "ebd66322437f5ba8894c07084c53a5ac"
server_key_iv: "98285e831445f8ab3463c3b58341b257"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.248359 21496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:12:56.248981 21516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.249125 21496 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:12:56.249205 21496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "6ded34b5968349b4a524a0357370d29f"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "ebd66322437f5ba8894c07084c53a5ac"
server_key_iv: "98285e831445f8ab3463c3b58341b257"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.249264 21496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:56.260650 21496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:56.262377 21496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:56.262511 21496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:56.262957 21496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:56.263013 21496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.263046 21496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:56.263063 21496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.269097 21496 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:45095
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:56.269166 21707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:45095 every 8 connection(s)
I20260808 02:12:56.269445 21496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:12:56.271965 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.263435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:46607 (local address 127.19.58.254:40063)
0808 02:12:56.263485 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.263487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.263687 (+ 200us) server_negotiation.cc:407] Connection header received
0808 02:12:56.264182 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.264184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.264196 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.264248 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:56.264963 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.265711 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.267087 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.267385 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.267451 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.269572 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:56.269574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:56.269574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:56.269582 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:56.270742 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.271094 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.271095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.271095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.271153 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.271360 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.271361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.271361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.271477 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:56.271533 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.271789 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.271872 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:12:56.272119 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.263404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40063 (local address 127.19.58.193:46607)
0808 02:12:56.263578 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.263611 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.264085 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.264309 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.264313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.264625 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:56.264879 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.264885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.265798 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.265799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.266995 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.266998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.267459 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.267460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.267587 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.268043 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:56.268053 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:56.269492 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:56.270813 (+ 1321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.270820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.270822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.271028 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.271206 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.271206 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.271207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.271306 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.271532 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:56.271538 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:56.271662 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.271769 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.271923 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260808 02:12:56.272586 21708 heartbeater.cc:347] Connected to a master server at 127.19.58.254:40063
I20260808 02:12:56.272672 21708 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:56.272826 21708 heartbeater.cc:511] Master 127.19.58.254:40063 requested a full tablet report, sending...
I20260808 02:12:56.273303 21391 ts_manager.cc:194] Registered new tserver with Master: 6ded34b5968349b4a524a0357370d29f (127.19.58.193:45095)
I20260808 02:12:56.274147 21391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:46607
I20260808 02:12:56.278039 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.273516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:38997 (local address 127.19.58.254:40063)
0808 02:12:56.273551 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.273553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.273665 (+ 112us) server_negotiation.cc:407] Connection header received
0808 02:12:56.273727 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.273728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.273734 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.273774 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:56.274082 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.274563 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.275313 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.275496 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.275545 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.276308 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:56.276309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:56.276309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:56.276314 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:56.277307 (+ 993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.277573 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.277574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.277574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.277614 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.277726 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.277726 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.277726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.277802 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:56.277841 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.277938 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.277987 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:12:56.278060 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.273470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40063 (local address 127.19.58.193:38997)
0808 02:12:56.273607 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.273619 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.273678 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.273782 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.273782 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.273968 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:56.274022 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.274023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.274644 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.274645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.275238 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.275240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.275548 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.275549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.275600 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.275967 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:56.275971 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:56.276247 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:56.277366 (+ 1119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.277366 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.277367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.277522 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.277649 (+ 127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.277650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.277650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.277695 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.277842 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:56.277843 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:56.277875 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.277915 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.277978 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260808 02:12:56.278664 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 21496
I20260808 02:12:56.278755 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:12:56.278921 19691 external_mini_cluster.cc:1428] Setting key c1fc490869557182a3662d2266798f86
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:12:56.300169 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40063
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:33913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:56.379479 21724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:56.379674 21724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:56.379727 21724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:56.380834 21724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:56.380872 21724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:56.380899 21724 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:56.380923 21724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:12:56.382294 21724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:33913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.58.254:40063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:56.382510 21724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:56.382706 21724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:56.387265 21730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:56.387255 21731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:56.387254 21735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:56.387332 21724 server_base.cc:1034] running on GCE node
I20260808 02:12:56.387669 21724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:56.387916 21724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:56.389052 21724 hybrid_clock.cc:648] HybridClock initialized: now 1786155176389037 us; error 29 us; skew 500 ppm
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:56.391311 21724 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:12:56.391727 21724 webserver.cc:460] Webserver started at http://127.19.58.194:41699/ using document root <none> and password file <none>
I20260808 02:12:56.391932 21724 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:56.391979 21724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:56.392104 21724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:56.393071 21724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "f0d6518dced540488f6ab5f2dd00ce25"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "923b900fc81283526cc036ca6cc320af"
server_key_iv: "b07062c20d8415fdb62e891aad1a9991"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.393405 21724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "f0d6518dced540488f6ab5f2dd00ce25"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "923b900fc81283526cc036ca6cc320af"
server_key_iv: "b07062c20d8415fdb62e891aad1a9991"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.394649 21724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:12:56.395278 21746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.395439 21724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:12:56.395524 21724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "f0d6518dced540488f6ab5f2dd00ce25"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "923b900fc81283526cc036ca6cc320af"
server_key_iv: "b07062c20d8415fdb62e891aad1a9991"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.395608 21724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:56.404739 21724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:56.406320 21724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:56.406447 21724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:56.406894 21724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:56.406932 21724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.406958 21724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:56.406975 21724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:56.413533 21724 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:34321
I20260808 02:12:56.413587 21914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:34321 every 8 connection(s)
I20260808 02:12:56.413923 21724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:12:56.414510 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 21724
I20260808 02:12:56.414604 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:12:56.414779 19691 external_mini_cluster.cc:1428] Setting key b811ba25e238a97846ea1ce046e90a85
I20260808 02:12:56.416739 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.407310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:42967 (local address 127.19.58.254:40063)
0808 02:12:56.407360 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.407362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.407565 (+ 203us) server_negotiation.cc:407] Connection header received
0808 02:12:56.408059 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.408061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.408073 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.408121 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:56.408843 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.409603 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.410984 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.411280 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.412349 (+ 1069us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.413956 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:56.413957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:56.413958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:56.413966 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:56.415467 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.415861 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.415862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.415863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.415929 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.416149 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.416150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.416150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.416259 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:56.416311 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.416569 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.416650 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:12:56.416859 21757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.407303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40063 (local address 127.19.58.194:42967)
0808 02:12:56.407453 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.407484 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.407954 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.408201 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.408205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.408467 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:56.408737 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.408743 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.409695 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.409696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.410893 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.410896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.411402 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.411403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.411546 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.412083 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:56.412094 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:56.413868 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:56.415541 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.415547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.415550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.415801 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.415982 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.415983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.415983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.416086 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.416310 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:56.416316 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:56.416448 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.416526 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.416669 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260808 02:12:56.417249 21915 heartbeater.cc:347] Connected to a master server at 127.19.58.254:40063
I20260808 02:12:56.417325 21915 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:56.417506 21915 heartbeater.cc:511] Master 127.19.58.254:40063 requested a full tablet report, sending...
I20260808 02:12:56.417825 21391 ts_manager.cc:194] Registered new tserver with Master: f0d6518dced540488f6ab5f2dd00ce25 (127.19.58.194:34321)
I20260808 02:12:56.418259 21391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:42967
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
I20260808 02:12:56.423949 21927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.418226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40063 (local address 127.19.58.194:43541)
0808 02:12:56.418357 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.418368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.418417 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.418608 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.418609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.418808 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:56.418863 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.418865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.419895 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.419895 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.420364 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.420366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.420791 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.420791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.420843 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.421213 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:56.421218 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:56.421468 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:56.422970 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.422970 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.422971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.423126 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.423312 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.423313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.423313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.423355 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.423637 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:56.423638 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:56.423670 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.423754 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.423831 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260808 02:12:56.423959 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.418456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:43541 (local address 127.19.58.254:40063)
0808 02:12:56.418492 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.418494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.418500 (+ 6us) server_negotiation.cc:407] Connection header received
0808 02:12:56.418517 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.418518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.418526 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.418569 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:56.418958 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.419798 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.420447 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.420723 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.420796 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.421546 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:56.421548 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:56.421548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:56.421555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:56.422894 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.423195 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.423196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.423196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.423251 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.423392 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.423392 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.423393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.423573 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:56.423638 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.423785 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.423866 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:12:56.439585 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40063
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:33913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:56.518940 21930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:56.519127 21930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:56.519152 21930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:56.520310 21930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:56.520349 21930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:56.520368 21930 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:56.520383 21930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:12:56.521708 21930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:33913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.58.254:40063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:56.521872 21930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:56.522034 21930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:56.526628 21938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:56.526741 21939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:56.526667 21942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:56.526736 21930 server_base.cc:1034] running on GCE node
I20260808 02:12:56.527002 21930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:56.527191 21930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:56.528314 21930 hybrid_clock.cc:648] HybridClock initialized: now 1786155176528295 us; error 38 us; skew 500 ppm
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:56.530572 21930 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:12:56.530929 21930 webserver.cc:460] Webserver started at http://127.19.58.195:44251/ using document root <none> and password file <none>
I20260808 02:12:56.531088 21930 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:56.531126 21930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:56.531212 21930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:56.532164 21930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "de4dc7358cd145498e8811cc751c585e"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "23bb2cd6e0e661e7e590748ca76bc579"
server_key_iv: "d40c3aaa1f72f7067480b4d06c83052b"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.532449 21930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "de4dc7358cd145498e8811cc751c585e"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "23bb2cd6e0e661e7e590748ca76bc579"
server_key_iv: "d40c3aaa1f72f7067480b4d06c83052b"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.533558 21930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:12:56.534209 21953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.534358 21930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:12:56.534441 21930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "de4dc7358cd145498e8811cc751c585e"
format_stamp: "Formatted at 2026-08-08 02:12:56 on dist-test-slave-1l3l"
server_key: "23bb2cd6e0e661e7e590748ca76bc579"
server_key_iv: "d40c3aaa1f72f7067480b4d06c83052b"
server_key_version: "encryptionkey@0"
I20260808 02:12:56.534498 21930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:56.548663 21930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:56.550168 21930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:56.550298 21930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:56.550757 21930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:56.550796 21930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.550823 21930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:56.550840 21930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.557256 21930 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:36643
I20260808 02:12:56.557330 22123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:36643 every 8 connection(s)
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:56.557605 21930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:12:56.560518 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.551151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:44483 (local address 127.19.58.254:40063)
0808 02:12:56.551215 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.551218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.551374 (+ 156us) server_negotiation.cc:407] Connection header received
0808 02:12:56.551844 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.551845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.551858 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.551957 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:56.552657 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.553399 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.554822 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.555143 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.555217 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.558117 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:56.558118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:56.558119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:56.558126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:56.559261 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.559586 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.559586 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.559586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.559649 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.559918 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.559919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.559919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.560034 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:56.560092 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.560346 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.560428 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:12:56.560642 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.551113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40063 (local address 127.19.58.195:44483)
0808 02:12:56.551257 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.551294 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.551742 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.552038 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.552041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.552300 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:56.552558 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.552563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.553487 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.553488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.554735 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.554738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.555237 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.555238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.555386 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.556421 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:56.556429 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:56.558047 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:56.559320 (+ 1273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.559326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.559328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.559529 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.559733 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.559734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.559734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.559834 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.560089 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:56.560096 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:56.560220 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.560297 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.560449 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260808 02:12:56.561069 22124 heartbeater.cc:347] Connected to a master server at 127.19.58.254:40063
I20260808 02:12:56.561149 22124 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:56.561304 22124 heartbeater.cc:511] Master 127.19.58.254:40063 requested a full tablet report, sending...
I20260808 02:12:56.561619 21390 ts_manager.cc:194] Registered new tserver with Master: de4dc7358cd145498e8811cc751c585e (127.19.58.195:36643)
I20260808 02:12:56.561991 21390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:44483
I20260808 02:12:56.564133 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 21930
I20260808 02:12:56.564213 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:12:56.564422 19691 external_mini_cluster.cc:1428] Setting key 099106fccacc4bcdcfba5ea68d41ef53
I20260808 02:12:56.565294 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:12:56.567397 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.562035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:59653 (local address 127.19.58.254:40063)
0808 02:12:56.562074 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.562076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.562210 (+ 134us) server_negotiation.cc:407] Connection header received
0808 02:12:56.562273 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.562273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.562281 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.562308 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:56.562907 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.563357 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.564064 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.564262 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.564312 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.565237 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:56.565238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:56.565238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:56.565244 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:56.566453 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.566739 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.566740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.566741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.566792 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.566954 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.566954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.566954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.567054 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:56.567113 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.567241 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.567303 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:12:56.567397 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.561959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40063 (local address 127.19.58.195:59653)
0808 02:12:56.562158 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.562169 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.562221 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.562665 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.562665 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.562804 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:56.562851 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.562853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.563425 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.563425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.564003 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.564006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.564330 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.564331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.564404 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.564824 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:56.564828 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:56.565169 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:56.566525 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.566526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.566527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.566680 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.566843 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.566844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.566844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.566906 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.567107 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:56.567108 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:56.567151 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.567211 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.567304 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:56.586912 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.581265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.19.58.254:40063)
0808 02:12:56.581302 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.581303 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.581311 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:12:56.581333 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.581334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.581341 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.581390 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:56.581888 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.582340 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.582980 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.583152 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.583200 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.585125 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:56.585126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:56.585126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:56.585132 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:56.586128 (+ 996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.586402 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.586402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.586403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.586444 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.586570 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.586571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.586571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.586652 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:56.586715 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.586785 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.586836 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:12:56.588259 21390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:56.589689 21390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:56.595381 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.591589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:34321 (local address 127.0.0.1:48070)
0808 02:12:56.591762 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.591801 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.591866 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.592105 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.592107 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.592109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:56.592174 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.592175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.593146 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.593147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.594244 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.594246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.595062 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.595063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.595169 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.595179 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.595238 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.595289 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":53,"threads_started":1}
I20260808 02:12:56.595409 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.591693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.19.58.194:34321)
0808 02:12:56.591937 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.591942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.591954 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:12:56.591999 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.592001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.592016 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.592104 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:56.592259 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.593068 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.594339 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.594978 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.595046 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.595107 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.595262 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.595313 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":129,"threads_started":1}
I20260808 02:12:56.595858 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.591608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:36643 (local address 127.0.0.1:44770)
0808 02:12:56.591974 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.591983 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.592021 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.592437 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.592438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.592441 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:56.592474 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.592475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.593655 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.593656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.594477 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.594480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.595569 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.595570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.595633 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.595637 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.595696 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.595759 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":54,"threads_started":1}
I20260808 02:12:56.596014 22150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.591595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45095 (local address 127.0.0.1:45492)
0808 02:12:56.591819 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.591828 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.591866 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.592222 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.592223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.592226 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:56.592285 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.592286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.593617 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.593618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.594647 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.594649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.595795 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.595796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.595860 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.595865 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.595904 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.595938 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":55,"threads_started":1}
I20260808 02:12:56.596091 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.591855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.19.58.193:45095)
0808 02:12:56.592074 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.592079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.592088 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:12:56.592123 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.592125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.592139 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.592219 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:56.592380 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.593527 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.594727 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.595690 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.595778 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.595885 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.595916 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.595987 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":105,"threads_started":1}
I20260808 02:12:56.596174 21811 tablet_service.cc:1467] Processing CreateTablet for tablet d1974fe08aa54c51a784e81b5fcb0243 (DEFAULT_TABLE table=test-table [id=d0a33d776a834b308cb5b605c93755cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:56.596568 21811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1974fe08aa54c51a784e81b5fcb0243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:56.596721 21602 tablet_service.cc:1467] Processing CreateTablet for tablet d1974fe08aa54c51a784e81b5fcb0243 (DEFAULT_TABLE table=test-table [id=d0a33d776a834b308cb5b605c93755cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:56.597050 21602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1974fe08aa54c51a784e81b5fcb0243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:56.600414 22159 tablet_bootstrap.cc:492] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: Bootstrap starting.
I20260808 02:12:56.600561 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.592052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44770 (local address 127.19.58.195:36643)
0808 02:12:56.592240 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.592244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.592252 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:12:56.592286 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.592288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.592310 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.592378 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:56.592542 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.593594 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.594540 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.595249 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.595345 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.595407 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.600412 (+ 5005us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.600454 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
I20260808 02:12:56.600793 22160 tablet_bootstrap.cc:492] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: Bootstrap starting.
I20260808 02:12:56.601192 22159 tablet_bootstrap.cc:654] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:56.601169 22023 tablet_service.cc:1467] Processing CreateTablet for tablet d1974fe08aa54c51a784e81b5fcb0243 (DEFAULT_TABLE table=test-table [id=d0a33d776a834b308cb5b605c93755cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:56.601426 22160 tablet_bootstrap.cc:654] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:56.601508 22023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1974fe08aa54c51a784e81b5fcb0243. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:56.601522 22159 log.cc:824] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:56.601769 22160 log.cc:824] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:56.602239 22159 tablet_bootstrap.cc:492] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: No bootstrap required, opened a new log
I20260808 02:12:56.602303 22159 ts_tablet_manager.cc:1397] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:12:56.602483 22160 tablet_bootstrap.cc:492] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: No bootstrap required, opened a new log
I20260808 02:12:56.602555 22160 ts_tablet_manager.cc:1397] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:12:56.604029 22159 raft_consensus.cc:348] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.604154 22159 raft_consensus.cc:374] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:56.604185 22159 raft_consensus.cc:729] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d6518dced540488f6ab5f2dd00ce25, State: Initialized, Role: FOLLOWER
I20260808 02:12:56.604312 22159 consensus_queue.cc:260] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.604473 22160 raft_consensus.cc:348] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.604584 21915 heartbeater.cc:503] Master 127.19.58.254:40063 was elected leader, sending a full tablet report...
I20260808 02:12:56.604627 22160 raft_consensus.cc:374] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:56.604681 22160 raft_consensus.cc:729] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ded34b5968349b4a524a0357370d29f, State: Initialized, Role: FOLLOWER
I20260808 02:12:56.604574 22159 ts_tablet_manager.cc:1428] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:56.604784 22160 consensus_queue.cc:260] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.605038 21708 heartbeater.cc:503] Master 127.19.58.254:40063 was elected leader, sending a full tablet report...
I20260808 02:12:56.605278 22160 ts_tablet_manager.cc:1428] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 02:12:56.605466 22168 tablet_bootstrap.cc:492] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: Bootstrap starting.
I20260808 02:12:56.606218 22168 tablet_bootstrap.cc:654] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:56.606463 22168 log.cc:824] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:56.607067 22168 tablet_bootstrap.cc:492] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: No bootstrap required, opened a new log
I20260808 02:12:56.607132 22168 ts_tablet_manager.cc:1397] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:12:56.608492 22168 raft_consensus.cc:348] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.608608 22168 raft_consensus.cc:374] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:56.608639 22168 raft_consensus.cc:729] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4dc7358cd145498e8811cc751c585e, State: Initialized, Role: FOLLOWER
I20260808 02:12:56.608738 22168 consensus_queue.cc:260] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.608961 22168 ts_tablet_manager.cc:1428] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:56.609011 22124 heartbeater.cc:503] Master 127.19.58.254:40063 was elected leader, sending a full tablet report...
W20260808 02:12:56.664525 21917 tablet.cc:2369] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:12:56.759692 22169 raft_consensus.cc:482] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:56.759843 22169 raft_consensus.cc:504] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.760195 22169 leader_election.cc:288] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ded34b5968349b4a524a0357370d29f (127.19.58.193:45095), de4dc7358cd145498e8811cc751c585e (127.19.58.195:36643)
I20260808 02:12:56.762970 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.760535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:44861 (local address 127.19.58.195:36643)
0808 02:12:56.760601 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.760602 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.760610 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:12:56.760634 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.760635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.760642 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.760694 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:56.760859 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.761354 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.762312 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.762658 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.762721 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.762729 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.762864 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.762898 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 02:12:56.762970 21757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.760395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:36643 (local address 127.19.58.194:44861)
0808 02:12:56.760481 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.760497 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.760542 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.760714 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.760715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.760717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:56.760790 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.760792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.761431 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.761432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.762241 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.762243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.762727 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.762728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.762805 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.762808 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.762854 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.762898 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 02:12:56.763170 22174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.760371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45095 (local address 127.19.58.194:34201)
0808 02:12:56.760673 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.760681 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.760730 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.760868 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.760868 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.760883 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:56.760981 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.760983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.761603 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.761604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.762387 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.762390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.762911 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.762912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.762977 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.762980 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.763025 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.763066 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":73,"thread_start_us":127,"threads_started":1}
I20260808 02:12:56.763263 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.760427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:34201 (local address 127.19.58.193:45095)
0808 02:12:56.760482 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.760484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.760741 (+ 257us) server_negotiation.cc:407] Connection header received
0808 02:12:56.760802 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.760803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.760812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.760859 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:56.761044 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.761520 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.762473 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.762834 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.762897 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.762905 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.763162 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.763203 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:12:56.763312 22059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1974fe08aa54c51a784e81b5fcb0243" candidate_uuid: "f0d6518dced540488f6ab5f2dd00ce25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4dc7358cd145498e8811cc751c585e" is_pre_election: true
I20260808 02:12:56.763460 22059 raft_consensus.cc:2454] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d6518dced540488f6ab5f2dd00ce25 in term 0.
I20260808 02:12:56.763581 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1974fe08aa54c51a784e81b5fcb0243" candidate_uuid: "f0d6518dced540488f6ab5f2dd00ce25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ded34b5968349b4a524a0357370d29f" is_pre_election: true
I20260808 02:12:56.763732 21749 leader_election.cc:302] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de4dc7358cd145498e8811cc751c585e, f0d6518dced540488f6ab5f2dd00ce25; no voters:
I20260808 02:12:56.763757 21636 raft_consensus.cc:2454] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d6518dced540488f6ab5f2dd00ce25 in term 0.
I20260808 02:12:56.763873 22169 raft_consensus.cc:2781] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:56.763952 22169 raft_consensus.cc:482] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:56.763976 22169 raft_consensus.cc:3037] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:56.765013 22169 raft_consensus.cc:504] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.765141 22169 leader_election.cc:288] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [CANDIDATE]: Term 1 election: Requested vote from peers 6ded34b5968349b4a524a0357370d29f (127.19.58.193:45095), de4dc7358cd145498e8811cc751c585e (127.19.58.195:36643)
I20260808 02:12:56.765314 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1974fe08aa54c51a784e81b5fcb0243" candidate_uuid: "f0d6518dced540488f6ab5f2dd00ce25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ded34b5968349b4a524a0357370d29f"
I20260808 02:12:56.765316 22059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1974fe08aa54c51a784e81b5fcb0243" candidate_uuid: "f0d6518dced540488f6ab5f2dd00ce25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4dc7358cd145498e8811cc751c585e"
I20260808 02:12:56.765398 22059 raft_consensus.cc:3037] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:56.765414 21636 raft_consensus.cc:3037] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:56.766268 22059 raft_consensus.cc:2454] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d6518dced540488f6ab5f2dd00ce25 in term 1.
I20260808 02:12:56.766268 21636 raft_consensus.cc:2454] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d6518dced540488f6ab5f2dd00ce25 in term 1.
I20260808 02:12:56.766462 21750 leader_election.cc:302] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ded34b5968349b4a524a0357370d29f, f0d6518dced540488f6ab5f2dd00ce25; no voters:
I20260808 02:12:56.766551 22169 raft_consensus.cc:2781] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:56.766705 22169 raft_consensus.cc:686] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 LEADER]: Becoming Leader. State: Replica: f0d6518dced540488f6ab5f2dd00ce25, State: Running, Role: LEADER
I20260808 02:12:56.766826 22169 consensus_queue.cc:237] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.767683 21391 catalog_manager.cc:5495] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0d6518dced540488f6ab5f2dd00ce25 (127.19.58.194). New cstate: current_term: 1 leader_uuid: "f0d6518dced540488f6ab5f2dd00ce25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } health_report { overall_health: UNKNOWN } } }
W20260808 02:12:56.770092 21709 tablet.cc:2369] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:12:56.774901 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.772527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.19.58.194:34321)
0808 02:12:56.772571 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.772572 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.772636 (+ 64us) server_negotiation.cc:407] Connection header received
0808 02:12:56.772718 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.772719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.772735 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.772787 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:56.772917 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.773380 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.774143 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.774316 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.774369 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.774592 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:56.774672 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:56.774738 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.774791 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.774808 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:12:56.776434 21636 raft_consensus.cc:1261] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f [term 1 FOLLOWER]: Refusing update from remote peer f0d6518dced540488f6ab5f2dd00ce25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:56.776444 22059 raft_consensus.cc:1261] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e [term 1 FOLLOWER]: Refusing update from remote peer f0d6518dced540488f6ab5f2dd00ce25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:56.776701 22175 consensus_queue.cc:1035] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:56.776814 22169 consensus_queue.cc:1035] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:56.779166 22178 mvcc.cc:205] Tried to move back new op lower bound from 7316091604074110976 to 7316091604037414912. Current Snapshot: MvccSnapshot[applied={T|T < 7316091604074110976}]
I20260808 02:12:56.779199 22180 mvcc.cc:205] Tried to move back new op lower bound from 7316091604074110976 to 7316091604037414912. Current Snapshot: MvccSnapshot[applied={T|T < 7316091604074110976}]
I20260808 02:12:56.779273 22181 mvcc.cc:205] Tried to move back new op lower bound from 7316091604074110976 to 7316091604037414912. Current Snapshot: MvccSnapshot[applied={T|T < 7316091604074110976}]
I20260808 02:12:56.782260 21388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:56.782352 21388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:56.783082 21388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2: Sending DeleteTablet for 3 replicas of tablet d1974fe08aa54c51a784e81b5fcb0243
I20260808 02:12:56.783485 21602 tablet_service.cc:1514] Processing DeleteTablet for tablet d1974fe08aa54c51a784e81b5fcb0243 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:56 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:45492
I20260808 02:12:56.783777 22190 tablet_replica.cc:330] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: stopping tablet replica
I20260808 02:12:56.784003 22190 raft_consensus.cc:2229] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:56.783982 21388 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:56.784096 21388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:56.784113 22190 raft_consensus.cc:2258] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:56.786415 21811 tablet_service.cc:1514] Processing DeleteTablet for tablet d1974fe08aa54c51a784e81b5fcb0243 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:56 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:48070
I20260808 02:12:56.786631 22191 tablet_replica.cc:330] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: stopping tablet replica
I20260808 02:12:56.786634 22023 tablet_service.cc:1514] Processing DeleteTablet for tablet d1974fe08aa54c51a784e81b5fcb0243 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:56 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:44770
I20260808 02:12:56.786733 22191 raft_consensus.cc:2229] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:12:56.786828 22191 raft_consensus.cc:2258] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:56.787204 22190 ts_tablet_manager.cc:1905] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:56.787882 21602 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc4947e1e6e4a14950e088d1a1ac029 (DEFAULT_TABLE table=test-table [id=3c79a199b3e14abd8057c7d34b73666f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:56.787926 22023 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc4947e1e6e4a14950e088d1a1ac029 (DEFAULT_TABLE table=test-table [id=3c79a199b3e14abd8057c7d34b73666f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:56.787993 21602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc4947e1e6e4a14950e088d1a1ac029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:56.788046 22023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc4947e1e6e4a14950e088d1a1ac029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:56.788060 21811 tablet_service.cc:1467] Processing CreateTablet for tablet 4fc4947e1e6e4a14950e088d1a1ac029 (DEFAULT_TABLE table=test-table [id=3c79a199b3e14abd8057c7d34b73666f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:56.788146 21811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fc4947e1e6e4a14950e088d1a1ac029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:56.789201 22160 tablet_bootstrap.cc:492] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f: Bootstrap starting.
I20260808 02:12:56.789338 22159 tablet_bootstrap.cc:492] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25: Bootstrap starting.
I20260808 02:12:56.789343 22168 tablet_bootstrap.cc:492] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: Bootstrap starting.
I20260808 02:12:56.789564 22160 tablet_bootstrap.cc:654] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:56.789741 22159 tablet_bootstrap.cc:654] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:56.790201 22191 ts_tablet_manager.cc:1905] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:56.790302 22159 tablet_bootstrap.cc:492] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25: No bootstrap required, opened a new log
I20260808 02:12:56.790351 22159 ts_tablet_manager.cc:1397] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:56.790503 22159 raft_consensus.cc:348] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.790560 22159 raft_consensus.cc:374] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:56.790578 22159 raft_consensus.cc:729] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d6518dced540488f6ab5f2dd00ce25, State: Initialized, Role: FOLLOWER
I20260808 02:12:56.790635 22159 consensus_queue.cc:260] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.790723 22159 ts_tablet_manager.cc:1428] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.791265 22190 ts_tablet_manager.cc:1918] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:12:56.791338 22190 log.cc:1196] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/d1974fe08aa54c51a784e81b5fcb0243
I20260808 02:12:56.791591 22190 ts_tablet_manager.cc:1939] T d1974fe08aa54c51a784e81b5fcb0243 P 6ded34b5968349b4a524a0357370d29f: Deleting consensus metadata
I20260808 02:12:56.791584 22191 ts_tablet_manager.cc:1918] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:12:56.791649 22191 log.cc:1196] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/d1974fe08aa54c51a784e81b5fcb0243
I20260808 02:12:56.791903 22191 ts_tablet_manager.cc:1939] T d1974fe08aa54c51a784e81b5fcb0243 P f0d6518dced540488f6ab5f2dd00ce25: Deleting consensus metadata
I20260808 02:12:56.792021 21369 catalog_manager.cc:4841] TS 6ded34b5968349b4a524a0357370d29f (127.19.58.193:45095): tablet d1974fe08aa54c51a784e81b5fcb0243 (table test-table [id=d0a33d776a834b308cb5b605c93755cf]) successfully deleted
I20260808 02:12:56.792063 22192 tablet_replica.cc:330] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: stopping tablet replica
I20260808 02:12:56.792150 22192 raft_consensus.cc:2229] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:56.792253 22192 raft_consensus.cc:2258] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:56.792253 21366 catalog_manager.cc:4841] TS f0d6518dced540488f6ab5f2dd00ce25 (127.19.58.194:34321): tablet d1974fe08aa54c51a784e81b5fcb0243 (table test-table [id=d0a33d776a834b308cb5b605c93755cf]) successfully deleted
I20260808 02:12:56.792368 22168 tablet_bootstrap.cc:654] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:56.792658 22192 ts_tablet_manager.cc:1905] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:56.793036 22168 tablet_bootstrap.cc:492] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: No bootstrap required, opened a new log
I20260808 02:12:56.793079 22168 ts_tablet_manager.cc:1397] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 02:12:56.793210 22168 raft_consensus.cc:348] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.793260 22168 raft_consensus.cc:374] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:56.793278 22168 raft_consensus.cc:729] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4dc7358cd145498e8811cc751c585e, State: Initialized, Role: FOLLOWER
I20260808 02:12:56.793319 22168 consensus_queue.cc:260] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.793406 22168 ts_tablet_manager.cc:1428] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.794154 22192 ts_tablet_manager.cc:1918] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:12:56.794221 22192 log.cc:1196] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/d1974fe08aa54c51a784e81b5fcb0243
I20260808 02:12:56.794456 22192 ts_tablet_manager.cc:1939] T d1974fe08aa54c51a784e81b5fcb0243 P de4dc7358cd145498e8811cc751c585e: Deleting consensus metadata
I20260808 02:12:56.794812 21368 catalog_manager.cc:4841] TS de4dc7358cd145498e8811cc751c585e (127.19.58.195:36643): tablet d1974fe08aa54c51a784e81b5fcb0243 (table test-table [id=d0a33d776a834b308cb5b605c93755cf]) successfully deleted
I20260808 02:12:56.804054 22160 tablet_bootstrap.cc:492] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f: No bootstrap required, opened a new log
I20260808 02:12:56.804111 22160 ts_tablet_manager.cc:1397] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.001s
I20260808 02:12:56.804418 22160 raft_consensus.cc:348] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.804494 22160 raft_consensus.cc:374] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:56.804513 22160 raft_consensus.cc:729] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ded34b5968349b4a524a0357370d29f, State: Initialized, Role: FOLLOWER
I20260808 02:12:56.804553 22160 consensus_queue.cc:260] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.804653 22160 ts_tablet_manager.cc:1428] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260808 02:12:56.808097 22125 tablet.cc:2369] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:12:56.876034 22171 raft_consensus.cc:482] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:56.876158 22171 raft_consensus.cc:504] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.876515 22171 leader_election.cc:288] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0d6518dced540488f6ab5f2dd00ce25 (127.19.58.194:34321), de4dc7358cd145498e8811cc751c585e (127.19.58.195:36643)
I20260808 02:12:56.880049 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.876910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:60093 (local address 127.19.58.195:36643)
0808 02:12:56.876954 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.876956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.876966 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:12:56.877035 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.877036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.877045 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.877112 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:56.877281 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.878043 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.879220 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.879662 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.879734 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.879745 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.879945 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.879980 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:12:56.880065 22197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.876674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:36643 (local address 127.19.58.193:60093)
0808 02:12:56.876918 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.876927 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.876974 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.877120 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.877121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.877124 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:56.877207 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.877209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.878151 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.878152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.879125 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.879128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.879806 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.879806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.879880 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.879884 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.879930 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.879968 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":75,"thread_start_us":87,"threads_started":1}
I20260808 02:12:56.880421 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.876814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:59863 (local address 127.19.58.194:34321)
0808 02:12:56.876862 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.876864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.876945 (+ 81us) server_negotiation.cc:407] Connection header received
0808 02:12:56.877105 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.877107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.877123 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.877173 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:56.877357 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.878042 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.879541 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.880035 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.880098 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.880106 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.880323 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.880362 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:12:56.880465 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.876674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:34321 (local address 127.19.58.193:59863)
0808 02:12:56.876846 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.876857 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.876984 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.877192 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.877193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.877196 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:56.877274 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.877276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.878140 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.878141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.879436 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.879447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.880165 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.880166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.880229 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.880231 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.880306 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.880394 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260808 02:12:56.880466 22059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc4947e1e6e4a14950e088d1a1ac029" candidate_uuid: "6ded34b5968349b4a524a0357370d29f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4dc7358cd145498e8811cc751c585e" is_pre_election: true
I20260808 02:12:56.880556 22059 raft_consensus.cc:2454] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ded34b5968349b4a524a0357370d29f in term 0.
I20260808 02:12:56.880726 21518 leader_election.cc:302] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ded34b5968349b4a524a0357370d29f, de4dc7358cd145498e8811cc751c585e; no voters:
I20260808 02:12:56.880707 21845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc4947e1e6e4a14950e088d1a1ac029" candidate_uuid: "6ded34b5968349b4a524a0357370d29f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d6518dced540488f6ab5f2dd00ce25" is_pre_election: true
I20260808 02:12:56.880823 21845 raft_consensus.cc:2454] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ded34b5968349b4a524a0357370d29f in term 0.
I20260808 02:12:56.880831 22171 raft_consensus.cc:2781] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:56.880858 22171 raft_consensus.cc:482] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:56.880875 22171 raft_consensus.cc:3037] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:56.881758 22171 raft_consensus.cc:504] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.881888 22171 leader_election.cc:288] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [CANDIDATE]: Term 1 election: Requested vote from peers f0d6518dced540488f6ab5f2dd00ce25 (127.19.58.194:34321), de4dc7358cd145498e8811cc751c585e (127.19.58.195:36643)
I20260808 02:12:56.882048 22059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc4947e1e6e4a14950e088d1a1ac029" candidate_uuid: "6ded34b5968349b4a524a0357370d29f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4dc7358cd145498e8811cc751c585e"
I20260808 02:12:56.882119 22059 raft_consensus.cc:3037] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:56.882838 22059 raft_consensus.cc:2454] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ded34b5968349b4a524a0357370d29f in term 1.
I20260808 02:12:56.882951 21845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fc4947e1e6e4a14950e088d1a1ac029" candidate_uuid: "6ded34b5968349b4a524a0357370d29f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d6518dced540488f6ab5f2dd00ce25"
I20260808 02:12:56.883020 21845 raft_consensus.cc:3037] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:56.883023 21518 leader_election.cc:302] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ded34b5968349b4a524a0357370d29f, de4dc7358cd145498e8811cc751c585e; no voters:
I20260808 02:12:56.883121 22171 raft_consensus.cc:2781] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:56.883281 22171 raft_consensus.cc:686] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 1 LEADER]: Becoming Leader. State: Replica: 6ded34b5968349b4a524a0357370d29f, State: Running, Role: LEADER
I20260808 02:12:56.883359 22171 consensus_queue.cc:237] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.883837 21845 raft_consensus.cc:2454] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ded34b5968349b4a524a0357370d29f in term 1.
I20260808 02:12:56.884095 21391 catalog_manager.cc:5495] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ded34b5968349b4a524a0357370d29f (127.19.58.193). New cstate: current_term: 1 leader_uuid: "6ded34b5968349b4a524a0357370d29f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:12:56 dist-test-slave-1l3l krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:56.926586 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.920341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40063 (local address 127.0.0.1:44762)
0808 02:12:56.920498 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:56.920509 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:56.920565 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:56.920674 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:56.920674 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:56.920974 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:56.921020 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.921022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.921679 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.921680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.922224 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:56.922226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.922564 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.922565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:56.922616 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.923069 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:56.923073 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:56.924396 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:56.925742 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.925750 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.925752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.925963 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.926110 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:56.926111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:56.926111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:56.926161 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:56.926325 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:56.926329 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:56.926367 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 02:12:56.926412 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 02:12:56.926496 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260808 02:12:56.926635 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:56.920396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.19.58.254:40063)
0808 02:12:56.920443 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:56.920445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:56.920559 (+ 114us) server_negotiation.cc:407] Connection header received
0808 02:12:56.920603 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:56.920604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:56.920612 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:56.920663 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:56.921082 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.921618 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.922327 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:56.922507 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:56.922560 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:56.924481 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:56.924482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:56.924482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:56.924488 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:56.925671 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.926021 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.926021 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.926022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.926066 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:56.926198 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:56.926199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:56.926199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:56.926285 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:56.926328 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:56.926437 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 02:12:56.926538 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":24}
I20260808 02:12:56.928323 21391 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:44762:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:12:56.928500 21391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:56.929821 21602 tablet_service.cc:1467] Processing CreateTablet for tablet a66b6d7f1c274635b3e6d9844e13ff38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fe0b0a543cbd4c5bbe078d2d13dd6f13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:56.929829 22023 tablet_service.cc:1467] Processing CreateTablet for tablet a66b6d7f1c274635b3e6d9844e13ff38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fe0b0a543cbd4c5bbe078d2d13dd6f13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:56.929978 21602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a66b6d7f1c274635b3e6d9844e13ff38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:56.929981 22023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a66b6d7f1c274635b3e6d9844e13ff38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:56.931077 22168 tablet_bootstrap.cc:492] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: Bootstrap starting.
I20260808 02:12:56.931166 21811 tablet_service.cc:1467] Processing CreateTablet for tablet a66b6d7f1c274635b3e6d9844e13ff38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fe0b0a543cbd4c5bbe078d2d13dd6f13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:56.931314 21811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a66b6d7f1c274635b3e6d9844e13ff38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:56.931540 22168 tablet_bootstrap.cc:654] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:56.931818 22160 tablet_bootstrap.cc:492] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: Bootstrap starting.
I20260808 02:12:56.932140 22160 tablet_bootstrap.cc:654] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:56.932617 22159 tablet_bootstrap.cc:492] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: Bootstrap starting.
I20260808 02:12:56.932840 22160 tablet_bootstrap.cc:492] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: No bootstrap required, opened a new log
I20260808 02:12:56.932904 22160 ts_tablet_manager.cc:1397] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:12:56.932999 22159 tablet_bootstrap.cc:654] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:56.933060 22168 tablet_bootstrap.cc:492] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: No bootstrap required, opened a new log
I20260808 02:12:56.933097 22168 ts_tablet_manager.cc:1397] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 02:12:56.933053 22160 raft_consensus.cc:348] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.933126 22160 raft_consensus.cc:374] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:56.933146 22160 raft_consensus.cc:729] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ded34b5968349b4a524a0357370d29f, State: Initialized, Role: FOLLOWER
I20260808 02:12:56.933194 22160 consensus_queue.cc:260] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.933223 22168 raft_consensus.cc:348] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.933282 22160 ts_tablet_manager.cc:1428] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.933291 22168 raft_consensus.cc:374] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:56.933310 22168 raft_consensus.cc:729] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4dc7358cd145498e8811cc751c585e, State: Initialized, Role: FOLLOWER
I20260808 02:12:56.933347 22168 consensus_queue.cc:260] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.933349 22171 tablet_replica.cc:439] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } } }
I20260808 02:12:56.933432 22171 tablet_replica.cc:442] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:56.933439 22168 ts_tablet_manager.cc:1428] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.933532 22173 tablet_replica.cc:439] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } } }
I20260808 02:12:56.933634 22173 tablet_replica.cc:442] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:56.933947 22159 tablet_bootstrap.cc:492] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: No bootstrap required, opened a new log
I20260808 02:12:56.933997 22159 ts_tablet_manager.cc:1397] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:56.934103 22159 raft_consensus.cc:348] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.934165 22159 raft_consensus.cc:374] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:56.934185 22159 raft_consensus.cc:729] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d6518dced540488f6ab5f2dd00ce25, State: Initialized, Role: FOLLOWER
I20260808 02:12:56.934233 22159 consensus_queue.cc:260] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:56.934321 22159 ts_tablet_manager.cc:1428] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:56.934378 22184 tablet_replica.cc:439] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } } }
I20260808 02:12:56.934478 22184 tablet_replica.cc:442] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:57.106711 22173 raft_consensus.cc:482] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:57.106828 22173 raft_consensus.cc:504] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:57.107179 22173 leader_election.cc:288] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ded34b5968349b4a524a0357370d29f (127.19.58.193:45095), f0d6518dced540488f6ab5f2dd00ce25 (127.19.58.194:34321)
I20260808 02:12:57.111114 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.107462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:49047 (local address 127.19.58.193:45095)
0808 02:12:57.107521 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:57.107523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:57.107603 (+ 80us) server_negotiation.cc:407] Connection header received
0808 02:12:57.107668 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:57.107669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:57.107678 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:57.107770 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:57.107976 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.108756 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.110203 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.110751 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.110821 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.110829 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:57.111009 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 02:12:57.111043 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:12:57.111114 22152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.107441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:47921 (local address 127.19.58.194:34321)
0808 02:12:57.107496 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:57.107498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:57.107604 (+ 106us) server_negotiation.cc:407] Connection header received
0808 02:12:57.107668 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:57.107669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:57.107678 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:57.107770 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:57.107983 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.108729 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.110202 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.110751 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.110822 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.110829 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:57.111008 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 02:12:57.111044 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:12:57.111166 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.107372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:34321 (local address 127.19.58.195:47921)
0808 02:12:57.107525 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:57.107537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:57.107609 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:57.107776 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:57.107777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:57.107779 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:57.107874 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:57.107875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.108818 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.108819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:57.110126 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:57.110130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.110831 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.110832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:57.110941 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.110945 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:12:57.110998 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 02:12:57.111047 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":103,"thread_start_us":44,"threads_started":1}
I20260808 02:12:57.111166 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.107372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45095 (local address 127.19.58.195:49047)
0808 02:12:57.107525 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:57.107537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:57.107609 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:57.107777 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:57.107777 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:57.107780 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:57.107873 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:57.107875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.108824 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.108825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:57.110126 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:57.110130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.110833 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.110834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:57.110940 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.110945 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:12:57.110998 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 02:12:57.111047 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":55,"thread_start_us":77,"threads_started":1}
I20260808 02:12:57.111505 21845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a66b6d7f1c274635b3e6d9844e13ff38" candidate_uuid: "de4dc7358cd145498e8811cc751c585e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d6518dced540488f6ab5f2dd00ce25" is_pre_election: true
I20260808 02:12:57.111507 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a66b6d7f1c274635b3e6d9844e13ff38" candidate_uuid: "de4dc7358cd145498e8811cc751c585e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ded34b5968349b4a524a0357370d29f" is_pre_election: true
I20260808 02:12:57.111600 21636 raft_consensus.cc:2454] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4dc7358cd145498e8811cc751c585e in term 0.
I20260808 02:12:57.111601 21845 raft_consensus.cc:2454] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de4dc7358cd145498e8811cc751c585e in term 0.
I20260808 02:12:57.111770 21959 leader_election.cc:302] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ded34b5968349b4a524a0357370d29f, de4dc7358cd145498e8811cc751c585e; no voters:
I20260808 02:12:57.111929 22173 raft_consensus.cc:2781] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:57.111972 22173 raft_consensus.cc:482] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:57.111992 22173 raft_consensus.cc:3037] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:57.112680 22173 raft_consensus.cc:504] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:57.112802 22173 leader_election.cc:288] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [CANDIDATE]: Term 1 election: Requested vote from peers 6ded34b5968349b4a524a0357370d29f (127.19.58.193:45095), f0d6518dced540488f6ab5f2dd00ce25 (127.19.58.194:34321)
I20260808 02:12:57.112950 21845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a66b6d7f1c274635b3e6d9844e13ff38" candidate_uuid: "de4dc7358cd145498e8811cc751c585e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d6518dced540488f6ab5f2dd00ce25"
I20260808 02:12:57.112974 21636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a66b6d7f1c274635b3e6d9844e13ff38" candidate_uuid: "de4dc7358cd145498e8811cc751c585e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ded34b5968349b4a524a0357370d29f"
I20260808 02:12:57.113020 21845 raft_consensus.cc:3037] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:57.113036 21636 raft_consensus.cc:3037] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:57.113576 21845 raft_consensus.cc:2454] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4dc7358cd145498e8811cc751c585e in term 1.
I20260808 02:12:57.113675 21636 raft_consensus.cc:2454] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de4dc7358cd145498e8811cc751c585e in term 1.
I20260808 02:12:57.113761 21954 leader_election.cc:302] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: de4dc7358cd145498e8811cc751c585e, f0d6518dced540488f6ab5f2dd00ce25; no voters:
I20260808 02:12:57.113852 22173 raft_consensus.cc:2781] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:57.113998 22173 raft_consensus.cc:686] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [term 1 LEADER]: Becoming Leader. State: Replica: de4dc7358cd145498e8811cc751c585e, State: Running, Role: LEADER
I20260808 02:12:57.114073 22173 consensus_queue.cc:237] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } }
I20260808 02:12:57.114404 22219 tablet_replica.cc:439] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: TxnStatusTablet state changed. Reason: New leader de4dc7358cd145498e8811cc751c585e. Latest consensus state: current_term: 1 leader_uuid: "de4dc7358cd145498e8811cc751c585e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } } }
I20260808 02:12:57.114495 22219 tablet_replica.cc:442] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:57.114668 22221 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:12:57.114753 22221 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:12:57.114702 21391 catalog_manager.cc:5495] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e reported cstate change: term changed from 0 to 1, leader changed from <none> to de4dc7358cd145498e8811cc751c585e (127.19.58.195). New cstate: current_term: 1 leader_uuid: "de4dc7358cd145498e8811cc751c585e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } health_report { overall_health: HEALTHY } } }
I20260808 02:12:57.159488 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.157120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44778 (local address 127.19.58.195:36643)
0808 02:12:57.157182 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:57.157184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:57.157208 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:12:57.157243 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:57.157244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:57.157265 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:57.157318 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:57.157450 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.157926 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.158728 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.158936 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.158992 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.159172 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:57.159272 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:57.159333 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:57.159387 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 02:12:57.159408 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:12:57.159502 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.157038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:36643 (local address 127.0.0.1:44778)
0808 02:12:57.157102 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:57.157114 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:57.157161 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:57.157328 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:57.157329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:57.157330 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:57.157384 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:57.157386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.157996 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.157997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:57.158649 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:57.158652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.158998 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.159000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:57.159070 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.159087 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:57.159325 (+ 238us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:57.159334 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 02:12:57.159374 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 02:12:57.159437 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 02:12:57.160617 21636 raft_consensus.cc:1261] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f [term 1 FOLLOWER]: Refusing update from remote peer de4dc7358cd145498e8811cc751c585e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:57.160758 21845 raft_consensus.cc:1261] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 FOLLOWER]: Refusing update from remote peer de4dc7358cd145498e8811cc751c585e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:57.160984 22219 consensus_queue.cc:1035] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:57.161116 22173 consensus_queue.cc:1035] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:57.161749 22171 tablet_replica.cc:439] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: TxnStatusTablet state changed. Reason: New leader de4dc7358cd145498e8811cc751c585e. Latest consensus state: current_term: 1 leader_uuid: "de4dc7358cd145498e8811cc751c585e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } } }
I20260808 02:12:57.161862 22171 tablet_replica.cc:442] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:57.161892 22184 tablet_replica.cc:439] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: TxnStatusTablet state changed. Reason: New leader de4dc7358cd145498e8811cc751c585e. Latest consensus state: current_term: 1 leader_uuid: "de4dc7358cd145498e8811cc751c585e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } } }
I20260808 02:12:57.161978 22184 tablet_replica.cc:442] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:57.162603 22173 tablet_replica.cc:439] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de4dc7358cd145498e8811cc751c585e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } } }
I20260808 02:12:57.162686 22173 tablet_replica.cc:442] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:57.162611 22219 tablet_replica.cc:439] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de4dc7358cd145498e8811cc751c585e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } } }
I20260808 02:12:57.162982 22219 tablet_replica.cc:442] T a66b6d7f1c274635b3e6d9844e13ff38 P de4dc7358cd145498e8811cc751c585e: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:57.163120 22184 tablet_replica.cc:439] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de4dc7358cd145498e8811cc751c585e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } } }
I20260808 02:12:57.163192 22184 tablet_replica.cc:442] T a66b6d7f1c274635b3e6d9844e13ff38 P f0d6518dced540488f6ab5f2dd00ce25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:57.163506 22171 tablet_replica.cc:439] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de4dc7358cd145498e8811cc751c585e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ded34b5968349b4a524a0357370d29f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45095 } } peers { permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 } } peers { permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 } } }
I20260808 02:12:57.163578 22171 tablet_replica.cc:442] T a66b6d7f1c274635b3e6d9844e13ff38 P 6ded34b5968349b4a524a0357370d29f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:57.166375 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.164212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45496 (local address 127.19.58.193:45095)
0808 02:12:57.164249 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:57.164251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:57.164272 (+ 21us) server_negotiation.cc:407] Connection header received
0808 02:12:57.164345 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:57.164346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:57.164368 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:57.164441 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:57.164533 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.164985 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.165702 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.165903 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.165958 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.166095 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:57.166192 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:57.166251 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:57.166290 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 02:12:57.166312 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:12:57.171322 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.168687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:36511 (local address 127.19.58.195:36643)
0808 02:12:57.168730 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:57.168731 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:57.168819 (+ 88us) server_negotiation.cc:407] Connection header received
0808 02:12:57.168875 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:57.168875 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:57.168882 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:57.168930 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:57.169107 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.169534 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.170577 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.170854 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.170950 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.171126 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:57.171150 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:57.171185 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:57.171263 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 02:12:57.171278 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:12:57.171393 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.168616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:36643 (local address 127.19.58.193:36511)
0808 02:12:57.168754 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:57.168764 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:57.168825 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:57.168935 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:57.168936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:57.168938 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:57.169046 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:57.169048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.169613 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.169614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:57.170497 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:57.170500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.170954 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.170955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:57.171025 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.171042 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:57.171209 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:57.171211 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:12:57.171261 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 02:12:57.171306 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260808 02:12:57.173568 22059 raft_consensus.cc:1261] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e [term 1 FOLLOWER]: Refusing update from remote peer 6ded34b5968349b4a524a0357370d29f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:57.173594 21845 raft_consensus.cc:1261] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 FOLLOWER]: Refusing update from remote peer 6ded34b5968349b4a524a0357370d29f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:57.173840 22198 consensus_queue.cc:1035] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0d6518dced540488f6ab5f2dd00ce25" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 34321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:57.173993 22198 consensus_queue.cc:1035] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4dc7358cd145498e8811cc751c585e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:57.181743 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.179235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:35147 (local address 127.19.58.193:45095)
0808 02:12:57.179273 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:57.179274 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:57.179370 (+ 96us) server_negotiation.cc:407] Connection header received
0808 02:12:57.179432 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:57.179433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:57.179441 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:57.179473 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:57.179602 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.180127 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.181165 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.181337 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.181386 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.181578 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:57.181599 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:57.181629 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:57.181684 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 02:12:57.181697 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 02:12:57.181816 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.179154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45095 (local address 127.19.58.195:35147)
0808 02:12:57.179300 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:57.179311 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:57.179373 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:57.179479 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:57.179479 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:57.179481 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:57.179546 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:57.179547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.180192 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.180193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:57.181088 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:57.181091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.181430 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.181431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:57.181493 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.181506 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:57.181634 (+ 128us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:57.181636 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:12:57.181682 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:12:57.181724 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260808 02:12:57.190814 21391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:57.190919 21391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:57.191505 21391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9e273501c054c439fbe8da2b4bb98e2: Sending DeleteTablet for 3 replicas of tablet 4fc4947e1e6e4a14950e088d1a1ac029
I20260808 02:12:57.191843 21602 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fc4947e1e6e4a14950e088d1a1ac029 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:57 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:45492
I20260808 02:12:57.191895 21811 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fc4947e1e6e4a14950e088d1a1ac029 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:57 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:48070
I20260808 02:12:57.191953 22190 tablet_replica.cc:330] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f: stopping tablet replica
I20260808 02:12:57.191977 22191 tablet_replica.cc:330] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25: stopping tablet replica
I20260808 02:12:57.192018 22191 raft_consensus.cc:2229] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:57.192016 22190 raft_consensus.cc:2229] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:12:57.192055 22191 raft_consensus.cc:2258] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:57.192097 22190 raft_consensus.cc:2258] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:57.192476 22191 ts_tablet_manager.cc:1905] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:57.194116 22191 ts_tablet_manager.cc:1918] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:57.194182 22191 log.cc:1196] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/4fc4947e1e6e4a14950e088d1a1ac029
I20260808 02:12:57.194393 22191 ts_tablet_manager.cc:1939] T 4fc4947e1e6e4a14950e088d1a1ac029 P f0d6518dced540488f6ab5f2dd00ce25: Deleting consensus metadata
I20260808 02:12:57.194697 21366 catalog_manager.cc:4841] TS f0d6518dced540488f6ab5f2dd00ce25 (127.19.58.194:34321): tablet 4fc4947e1e6e4a14950e088d1a1ac029 (table test-table [id=3c79a199b3e14abd8057c7d34b73666f]) successfully deleted
I20260808 02:12:57.195008 22190 ts_tablet_manager.cc:1905] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:57.196651 22190 ts_tablet_manager.cc:1918] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:57.196714 22190 log.cc:1196] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/4fc4947e1e6e4a14950e088d1a1ac029
I20260808 02:12:57.196925 22190 ts_tablet_manager.cc:1939] T 4fc4947e1e6e4a14950e088d1a1ac029 P 6ded34b5968349b4a524a0357370d29f: Deleting consensus metadata
I20260808 02:12:57.197221 22023 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fc4947e1e6e4a14950e088d1a1ac029 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:57 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:44770
I20260808 02:12:57.197317 22192 tablet_replica.cc:330] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: stopping tablet replica
I20260808 02:12:57.197325 21369 catalog_manager.cc:4841] TS 6ded34b5968349b4a524a0357370d29f (127.19.58.193:45095): tablet 4fc4947e1e6e4a14950e088d1a1ac029 (table test-table [id=3c79a199b3e14abd8057c7d34b73666f]) successfully deleted
I20260808 02:12:57.197367 22192 raft_consensus.cc:2229] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:57.197409 22192 raft_consensus.cc:2258] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:57.197707 22192 ts_tablet_manager.cc:1905] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:57.199026 22192 ts_tablet_manager.cc:1918] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:57.199088 22192 log.cc:1196] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/4fc4947e1e6e4a14950e088d1a1ac029
I20260808 02:12:57.199306 22192 ts_tablet_manager.cc:1939] T 4fc4947e1e6e4a14950e088d1a1ac029 P de4dc7358cd145498e8811cc751c585e: Deleting consensus metadata
I20260808 02:12:57.199606 21368 catalog_manager.cc:4841] TS de4dc7358cd145498e8811cc751c585e (127.19.58.195:36643): tablet 4fc4947e1e6e4a14950e088d1a1ac029 (table test-table [id=3c79a199b3e14abd8057c7d34b73666f]) successfully deleted
Aug 08 02:12:57 dist-test-slave-1l3l krb5kdc[21327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155176, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:12:57.209625 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.198094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45510 (local address 127.19.58.193:45095)
0808 02:12:57.198129 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:57.198130 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:57.199881 (+ 1751us) server_negotiation.cc:407] Connection header received
0808 02:12:57.199931 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:57.199932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:57.199942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:57.199996 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:57.200655 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.201465 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.202550 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.202843 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.202918 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.206347 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:57.206355 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:57.206355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:57.206363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:57.208356 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:57.208755 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:57.208758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:57.208758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:57.208818 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:57.209047 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:57.209048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:57.209048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:57.209211 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:57.209361 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:57.209441 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 02:12:57.209528 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 02:12:57.210282 21698 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45510
I20260808 02:12:57.219131 21463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:57.211977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.19.58.254:40063)
0808 02:12:57.212026 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:57.212028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:57.212155 (+ 127us) server_negotiation.cc:407] Connection header received
0808 02:12:57.212207 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:57.212208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:57.212220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:57.212275 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:57.212853 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.213591 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.214581 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:57.214882 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:57.214954 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:57.216332 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:57.216334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:57.216335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:57.216343 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:57.218004 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:57.218378 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:57.218379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:57.218380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:57.218445 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:57.218628 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:57.218629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:57.218629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:57.218757 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:57.218861 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:57.218961 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 02:12:57.219037 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 02:12:57.219623 21391 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44776
I20260808 02:12:57.220368 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 21496
I20260808 02:12:57.231554 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 21724
I20260808 02:12:57.244017 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 21930
I20260808 02:12:57.253880 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 21348
2026-08-08T02:12:57Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2517 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:12:57 dist-test-slave-1l3l krb5kdc[22254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:12:57 dist-test-slave-1l3l krb5kdc[22254](info): set up 2 sockets
Aug 08 02:12:57 dist-test-slave-1l3l krb5kdc[22254](info): commencing operation
krb5kdc: starting...
W20260808 02:12:58.376919 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:12:58 dist-test-slave-1l3l krb5kdc[22254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:12:58.491518 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:44757
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:36619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:44757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:58.570634 22278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:58.570828 22278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:58.570850 22278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:58.572005 22278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:12:58.572042 22278 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:12:58.572055 22278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:58.572067 22278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:58.572080 22278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:12:58.572104 22278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:12:58.573542 22278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:36619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:44757
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:44757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:58.573762 22278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:58.573976 22278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:12:58.578649 22278 server_base.cc:1034] running on GCE node
W20260808 02:12:58.578625 22290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:58.578624 22287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:58.578679 22285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:58.580024 22278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:58.580293 22278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:58.581475 22278 hybrid_clock.cc:648] HybridClock initialized: now 1786155178581456 us; error 32 us; skew 500 ppm
Aug 08 02:12:58 dist-test-slave-1l3l krb5kdc[22254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:58.583897 22278 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:12:58.584360 22278 webserver.cc:460] Webserver started at http://127.19.58.254:39599/ using document root <none> and password file <none>
I20260808 02:12:58.584560 22278 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:58.584609 22278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:58.584736 22278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:58.585779 22278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-1l3l"
server_key: "70f33bac009b5d186f95031ab373af22"
server_key_iv: "30e1e027a8ed58e278cab280d25d1f57"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.586117 22278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-1l3l"
server_key: "70f33bac009b5d186f95031ab373af22"
server_key_iv: "30e1e027a8ed58e278cab280d25d1f57"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.587380 22278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:12:58.588097 22297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.588243 22278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.588362 22278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-1l3l"
server_key: "70f33bac009b5d186f95031ab373af22"
server_key_iv: "30e1e027a8ed58e278cab280d25d1f57"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.588423 22278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:58.600229 22278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:58.602244 22278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:58.602381 22278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:58.606838 22278 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:44757
I20260808 02:12:58.606904 22405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:44757 every 8 connection(s)
I20260808 02:12:58.607182 22278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:12:58.607816 22406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:58.610106 22406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86: Bootstrap starting.
I20260808 02:12:58.610694 22406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:58.610951 22406 log.cc:824] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:58.611598 22406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86: No bootstrap required, opened a new log
I20260808 02:12:58.613746 22406 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44757 } }
I20260808 02:12:58.613878 22406 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:58.613909 22406 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d5e4c05ebd4615a826cb12fe6b3f86, State: Initialized, Role: FOLLOWER
I20260808 02:12:58.614032 22406 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44757 } }
I20260808 02:12:58.614090 22406 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:12:58.614120 22406 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:12:58.614163 22406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:58.614660 22406 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44757 } }
I20260808 02:12:58.614753 22406 leader_election.cc:302] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4d5e4c05ebd4615a826cb12fe6b3f86; no voters:
I20260808 02:12:58.614899 22406 leader_election.cc:288] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:12:58.614933 22409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:58.615085 22409 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [term 1 LEADER]: Becoming Leader. State: Replica: f4d5e4c05ebd4615a826cb12fe6b3f86, State: Running, Role: LEADER
I20260808 02:12:58.615121 22406 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:12:58.615180 22409 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44757 } }
I20260808 02:12:58.615595 22411 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44757 } } }
I20260808 02:12:58.615690 22411 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:58.615604 22410 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4d5e4c05ebd4615a826cb12fe6b3f86. Latest consensus state: current_term: 1 leader_uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4d5e4c05ebd4615a826cb12fe6b3f86" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44757 } } }
I20260808 02:12:58.615849 22410 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86 [sys.catalog]: This master's current role is: LEADER
I20260808 02:12:58.615913 22426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:12:58.616030 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 22278
I20260808 02:12:58.616117 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:12:58.616294 19691 external_mini_cluster.cc:1428] Setting key 5ad911862ab1773245bf293099598508
I20260808 02:12:58.616433 22426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:12:58.618260 22426 catalog_manager.cc:1347] Generated new cluster ID: b4ea818eecb54820b1584ad17870770a
I20260808 02:12:58.618309 22426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:12:58 dist-test-slave-1l3l krb5kdc[22254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:58.627458 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.616869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.19.58.254:44757)
0808 02:12:58.617034 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:58.617043 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:58.617073 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:12:58.617233 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:58.617241 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:58.617294 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:58.617421 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:58.618128 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.619478 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.620606 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.621194 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.621285 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.624047 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:58.624051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:58.624052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:58.624064 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:58.625952 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.626300 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.626302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.626303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.626403 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.626572 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.626572 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.626573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.626875 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:58.627028 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:58.627106 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 02:12:58.627236 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260808 02:12:58.636804 22426 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:12:58.637259 22426 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:12:58.648947 22426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86: Generated new TSK 0
I20260808 02:12:58.649094 22426 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:12:58.674535 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:44757
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:36619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:58.753109 22456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:58.753297 22456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:58.753319 22456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:58.754366 22456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:58.754405 22456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:58.754423 22456 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:12:58.754441 22456 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:58.754462 22456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:12:58.755827 22456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:36619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.58.254:44757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:58.755991 22456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:58.756160 22456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:58.760663 22464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:58.760673 22463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:58.760756 22456 server_base.cc:1034] running on GCE node
W20260808 02:12:58.760735 22468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:58.761080 22456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:58.761269 22456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:58.762409 22456 hybrid_clock.cc:648] HybridClock initialized: now 1786155178762395 us; error 32 us; skew 500 ppm
Aug 08 02:12:58 dist-test-slave-1l3l krb5kdc[22254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:58.764696 22456 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:12:58.765081 22456 webserver.cc:460] Webserver started at http://127.19.58.193:33113/ using document root <none> and password file <none>
I20260808 02:12:58.765266 22456 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:58.765313 22456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:58.765444 22456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:58.766408 22456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "a449314c494146dc83c02a952a742c9c"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-1l3l"
server_key: "a868dad13552170a60c20c195095e2af"
server_key_iv: "e6e1c7398e0b7b318220a0646461a20d"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.766734 22456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "a449314c494146dc83c02a952a742c9c"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-1l3l"
server_key: "a868dad13552170a60c20c195095e2af"
server_key_iv: "e6e1c7398e0b7b318220a0646461a20d"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.768077 22456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:12:58.768764 22479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.768925 22456 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:12:58.769006 22456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "a449314c494146dc83c02a952a742c9c"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-1l3l"
server_key: "a868dad13552170a60c20c195095e2af"
server_key_iv: "e6e1c7398e0b7b318220a0646461a20d"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.769065 22456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:58.788894 22456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:58.790572 22456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:58.790704 22456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:58.791225 22456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:58.791265 22456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.791293 22456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:58.791311 22456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:12:58 dist-test-slave-1l3l krb5kdc[22254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:58.797632 22456 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:45537
I20260808 02:12:58.797756 22650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:45537 every 8 connection(s)
I20260808 02:12:58.797986 22456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:12:58.799198 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 22456
I20260808 02:12:58.799283 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:12:58.799443 19691 external_mini_cluster.cc:1428] Setting key 8242f0fb1f783d204ae826337abfc885
I20260808 02:12:58.800897 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.791641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:54259 (local address 127.19.58.254:44757)
0808 02:12:58.791680 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:58.791682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:58.791917 (+ 235us) server_negotiation.cc:407] Connection header received
0808 02:12:58.792373 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:58.792374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:58.792393 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:58.792440 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:58.793178 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.793935 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.795362 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.795680 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.795782 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.798135 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:58.798137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:58.798137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:58.798144 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:58.799485 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.799898 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.799899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.799899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.799950 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.800211 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.800212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.800212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.800350 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:58.800445 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:58.800728 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 02:12:58.800813 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 02:12:58.800977 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.791639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44757 (local address 127.19.58.193:54259)
0808 02:12:58.791805 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:58.791839 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:58.792283 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:58.792518 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:58.792522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:58.792819 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:58.793075 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.793080 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.794033 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.794034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.795252 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.795255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.795789 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.795790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.795935 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.796395 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:58.796405 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:58.798050 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:58.799553 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.799562 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.799564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.799836 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.800014 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.800015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.800015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.800147 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.800471 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:58.800478 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:58.800597 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 02:12:58.800694 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 02:12:58.800815 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260808 02:12:58.801558 22654 heartbeater.cc:347] Connected to a master server at 127.19.58.254:44757
I20260808 02:12:58.801693 22654 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:58.801944 22654 heartbeater.cc:511] Master 127.19.58.254:44757 requested a full tablet report, sending...
I20260808 02:12:58.802501 22337 ts_manager.cc:194] Registered new tserver with Master: a449314c494146dc83c02a952a742c9c (127.19.58.193:45537)
I20260808 02:12:58.803359 22337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:54259
I20260808 02:12:58.808108 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.802577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44757 (local address 127.19.58.193:38887)
0808 02:12:58.802749 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:58.802759 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:58.802815 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:58.802932 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:58.802932 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:58.803115 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:58.803168 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.803170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.804021 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.804022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.804578 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.804580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.805006 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.805007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.805057 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.805446 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:58.805450 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:58.805715 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:58.807241 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.807241 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.807242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.807411 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.807601 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.807602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.807602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.807647 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.807890 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:58.807891 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:58.807926 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 02:12:58.807974 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 02:12:58.808046 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":68,"threads_started":1}
I20260808 02:12:58.808166 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.802640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:38887 (local address 127.19.58.254:44757)
0808 02:12:58.802683 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:58.802685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:58.802809 (+ 124us) server_negotiation.cc:407] Connection header received
0808 02:12:58.802862 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:58.802863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:58.802874 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:58.802928 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:58.803239 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.803919 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.804655 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.804935 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.805011 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.805799 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:58.805801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:58.805801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:58.805809 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:58.807166 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.807481 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.807482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.807482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.807542 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.807684 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.807684 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.807685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.807832 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:58.807897 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:58.808001 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 02:12:58.808078 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:12:58.824025 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:44757
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:36619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:58.903136 22670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:58.903321 22670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:58.903343 22670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:58.904420 22670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:58.904453 22670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:58.904472 22670 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:12:58.904484 22670 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:58.904498 22670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:12:58.905836 22670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:36619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.58.254:44757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:58.906025 22670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:58.906195 22670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:58.910826 22676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:58.910899 22670 server_base.cc:1034] running on GCE node
W20260808 02:12:58.910799 22682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:58.910810 22675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:58.911195 22670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:58.911402 22670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:58.912531 22670 hybrid_clock.cc:648] HybridClock initialized: now 1786155178912518 us; error 25 us; skew 500 ppm
Aug 08 02:12:58 dist-test-slave-1l3l krb5kdc[22254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:58.914799 22670 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:12:58.915227 22670 webserver.cc:460] Webserver started at http://127.19.58.194:39043/ using document root <none> and password file <none>
I20260808 02:12:58.915458 22670 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:58.915506 22670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:58.915632 22670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:58.916640 22670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-1l3l"
server_key: "8dd3a32cb3f3f113795a83951d105d42"
server_key_iv: "25e456143a040d0c1f12532a43f5cfc8"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.916965 22670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-1l3l"
server_key: "8dd3a32cb3f3f113795a83951d105d42"
server_key_iv: "25e456143a040d0c1f12532a43f5cfc8"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.918155 22670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:12:58.918777 22690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.918934 22670 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:12:58.919015 22670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9"
format_stamp: "Formatted at 2026-08-08 02:12:58 on dist-test-slave-1l3l"
server_key: "8dd3a32cb3f3f113795a83951d105d42"
server_key_iv: "25e456143a040d0c1f12532a43f5cfc8"
server_key_version: "encryptionkey@0"
I20260808 02:12:58.919080 22670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:58.936370 22670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:58.938040 22670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:58.938169 22670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:58.938640 22670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:58.938681 22670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:58.938709 22670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:58.938733 22670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:12:58 dist-test-slave-1l3l krb5kdc[22254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:58.945418 22670 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:38095
I20260808 02:12:58.945724 22670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:12:58.945755 22872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:38095 every 8 connection(s)
I20260808 02:12:58.947623 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.938993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:39663 (local address 127.19.58.254:44757)
0808 02:12:58.939056 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:58.939058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:58.939241 (+ 183us) server_negotiation.cc:407] Connection header received
0808 02:12:58.939864 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:58.939865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:58.939886 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:58.939921 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:58.940746 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.941274 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.942253 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.942463 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.942518 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.945027 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:58.945028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:58.945029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:58.945037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:58.946398 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.946742 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.946743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.946743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.946797 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.947025 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.947026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.947027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.947136 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:58.947192 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:58.947452 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 02:12:58.947533 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:12:58.947775 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.938955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44757 (local address 127.19.58.194:39663)
0808 02:12:58.939107 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:58.939152 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:58.939758 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:58.940014 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:58.940018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:58.940360 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:58.940642 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.940648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.941360 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.941361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.942166 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.942168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.942533 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.942534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.942651 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.943067 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:58.943076 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:58.944942 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:58.946468 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.946474 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.946476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.946675 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.946851 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.946851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.946852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.946968 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.947191 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:58.947198 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:58.947320 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 02:12:58.947414 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 02:12:58.947557 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
I20260808 02:12:58.948287 22875 heartbeater.cc:347] Connected to a master server at 127.19.58.254:44757
I20260808 02:12:58.948386 22875 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:58.948518 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 22670
I20260808 02:12:58.948560 22875 heartbeater.cc:511] Master 127.19.58.254:44757 requested a full tablet report, sending...
I20260808 02:12:58.948602 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:12:58.948791 19691 external_mini_cluster.cc:1428] Setting key a7f9890699d9db395370a9bf373a7768
I20260808 02:12:58.948863 22338 ts_manager.cc:194] Registered new tserver with Master: ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194:38095)
I20260808 02:12:58.949323 22338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:39663
I20260808 02:12:58.954591 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.949301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:51577 (local address 127.19.58.254:44757)
0808 02:12:58.949335 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:58.949336 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:58.949466 (+ 130us) server_negotiation.cc:407] Connection header received
0808 02:12:58.949568 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:58.949569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:58.949581 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:58.949638 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:58.949968 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.950558 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.951307 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.951493 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.951541 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.952345 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:58.952345 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:58.952346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:58.952351 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:58.953612 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.953960 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.953961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.953961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.954005 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:58.954171 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:58.954171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:58.954171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:58.954289 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:58.954335 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:58.954485 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 02:12:58.954537 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 02:12:58.954669 22881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:58.949246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44757 (local address 127.19.58.194:51577)
0808 02:12:58.949410 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:58.949419 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:58.949489 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:58.949640 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:58.949640 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:58.949824 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:58.949895 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.949897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.950647 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.950648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.951233 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:58.951235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:58.951553 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:58.951554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:58.951621 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:58.951985 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:58.951989 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:58.952277 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:58.953696 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.953697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.953697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.953898 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.954057 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:58.954057 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:58.954058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:58.954119 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:58.954344 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:58.954345 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:58.954397 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 02:12:58.954470 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 02:12:58.954562 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:12:58.972111 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:44757
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:36619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:12:59.054301 22885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:12:59.054497 22885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:12:59.054518 22885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:12:59.055629 22885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:12:59.055668 22885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:12:59.055687 22885 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 02:12:59.055698 22885 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:12:59.055747 22885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:12:59.057183 22885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:36619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.58.254:44757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:12:59.057396 22885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:12:59.057602 22885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:12:59.062325 22892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:59.062362 22895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:12:59.062362 22893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:12:59.062520 22885 server_base.cc:1034] running on GCE node
I20260808 02:12:59.062760 22885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:12:59.062960 22885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:12:59.064085 22885 hybrid_clock.cc:648] HybridClock initialized: now 1786155179064076 us; error 31 us; skew 500 ppm
Aug 08 02:12:59 dist-test-slave-1l3l krb5kdc[22254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155179, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:12:59.066366 22885 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:12:59.066761 22885 webserver.cc:460] Webserver started at http://127.19.58.195:33185/ using document root <none> and password file <none>
I20260808 02:12:59.066926 22885 fs_manager.cc:359] Metadata directory not provided
I20260808 02:12:59.066965 22885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:12:59.067051 22885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:12:59.068013 22885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "456336d0fb6841c8bd4b9e2ef0ecc607"
format_stamp: "Formatted at 2026-08-08 02:12:59 on dist-test-slave-1l3l"
server_key: "76e49e190df9150b9a2c988ec772a8bd"
server_key_iv: "a982cd94c35f04f73cee5919ba0ba6ae"
server_key_version: "encryptionkey@0"
I20260808 02:12:59.068301 22885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "456336d0fb6841c8bd4b9e2ef0ecc607"
format_stamp: "Formatted at 2026-08-08 02:12:59 on dist-test-slave-1l3l"
server_key: "76e49e190df9150b9a2c988ec772a8bd"
server_key_iv: "a982cd94c35f04f73cee5919ba0ba6ae"
server_key_version: "encryptionkey@0"
I20260808 02:12:59.069465 22885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:59.070072 22907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:59.070212 22885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:12:59.070288 22885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "456336d0fb6841c8bd4b9e2ef0ecc607"
format_stamp: "Formatted at 2026-08-08 02:12:59 on dist-test-slave-1l3l"
server_key: "76e49e190df9150b9a2c988ec772a8bd"
server_key_iv: "a982cd94c35f04f73cee5919ba0ba6ae"
server_key_version: "encryptionkey@0"
I20260808 02:12:59.070346 22885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:12:59.079767 22885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:12:59.081274 22885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:12:59.081408 22885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:12:59.081858 22885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:12:59.081899 22885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:59.081925 22885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:12:59.081951 22885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:12:59 dist-test-slave-1l3l krb5kdc[22254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155179, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:59.088871 22885 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:34081
I20260808 02:12:59.088939 23092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:34081 every 8 connection(s)
I20260808 02:12:59.089236 22885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:12:59.091027 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.082197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:52451 (local address 127.19.58.254:44757)
0808 02:12:59.082245 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.082247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.082452 (+ 205us) server_negotiation.cc:407] Connection header received
0808 02:12:59.082889 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.082890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.082905 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.082963 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:59.083570 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.084179 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.085148 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.085345 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.085407 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.088275 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:59.088277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:59.088277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:59.088285 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:59.089764 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:59.090153 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:59.090154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:59.090154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:59.090210 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:59.090443 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:59.090444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:59.090444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:59.090555 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:59.090610 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.090852 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.090940 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:12:59.091180 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.082194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44757 (local address 127.19.58.195:52451)
0808 02:12:59.082337 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.082367 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.082786 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.083022 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.083025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.083280 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:59.083482 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.083487 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.084258 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.084259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.085063 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.085066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.085413 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.085413 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.085545 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.086022 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:59.086031 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:59.088191 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:59.089843 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:59.089849 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:59.089851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:59.090086 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:59.090262 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:59.090263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:59.090263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:59.090380 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:59.090609 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:59.090615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:59.090737 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.090821 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.090961 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260808 02:12:59.091579 23094 heartbeater.cc:347] Connected to a master server at 127.19.58.254:44757
I20260808 02:12:59.091655 23094 heartbeater.cc:464] Registering TS with master...
I20260808 02:12:59.091836 23094 heartbeater.cc:511] Master 127.19.58.254:44757 requested a full tablet report, sending...
I20260808 02:12:59.092422 22337 ts_manager.cc:194] Registered new tserver with Master: 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081)
I20260808 02:12:59.092810 22337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:52451
I20260808 02:12:59.096356 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 22885
I20260808 02:12:59.096450 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:12:59.096632 19691 external_mini_cluster.cc:1428] Setting key 5cceb43327d33f21b006b2a4ed588297
I20260808 02:12:59.097430 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:12:59.097656 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.092552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:37781 (local address 127.19.58.254:44757)
0808 02:12:59.092588 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.092590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.093030 (+ 440us) server_negotiation.cc:407] Connection header received
0808 02:12:59.093072 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.093073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.093082 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.093140 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:59.093450 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.093908 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.094534 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.094698 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.094737 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.095422 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:59.095423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:59.095423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:59.095428 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:59.096638 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:59.096966 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:59.096967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:59.096967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:59.097026 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:59.097215 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:59.097216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:59.097216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:59.097335 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:59.097396 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.097536 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.097600 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 02:12:59.097683 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.092502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44757 (local address 127.19.58.195:37781)
0808 02:12:59.092964 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.092974 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.093029 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.093143 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.093143 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.093328 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:59.093387 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.093389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.093990 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.093990 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.094477 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.094479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.094741 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.094742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.094790 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.095097 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:59.095100 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:59.095344 (+ 244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:59.096711 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:59.096712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:59.096712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:59.096893 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:59.097087 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:59.097088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:59.097088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:59.097156 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:59.097391 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:59.097392 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:59.097435 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.097503 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.097591 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":70,"threads_started":1}
Aug 08 02:12:59 dist-test-slave-1l3l krb5kdc[22254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:12:59 dist-test-slave-1l3l krb5kdc[22254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:59.119470 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.113509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45048 (local address 127.19.58.254:44757)
0808 02:12:59.113558 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.113559 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.113566 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:12:59.113584 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.113585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.113593 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.113639 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:59.114138 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.114625 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.115329 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.115507 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.115552 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.117607 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:59.117609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:59.117609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:59.117615 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:59.118747 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:59.118991 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:59.118991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:59.118992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:59.119033 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:59.119146 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:59.119146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:59.119146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:59.119225 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:59.119287 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.119359 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.119412 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 02:12:59.120774 22337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:59.122339 22337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:59.128257 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.124690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.19.58.194:38095)
0808 02:12:59.124862 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.124866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.124874 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:12:59.124909 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.124911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.124993 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.125066 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:59.125248 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.126311 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.127286 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.127874 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.127937 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.127994 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.128123 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.128159 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":76,"threads_started":1}
I20260808 02:12:59.128264 23113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.124428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:38095 (local address 127.0.0.1:47792)
0808 02:12:59.124621 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.124630 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.124673 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.125130 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.125131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.125133 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:59.125173 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.125175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.126381 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.126382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.127204 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.127207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.127947 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.127948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.128024 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.128038 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.128102 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.128162 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":3,"thread_start_us":50,"threads_started":1}
I20260808 02:12:59.128279 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.124194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45537 (local address 127.0.0.1:46454)
0808 02:12:59.124373 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.124383 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.124443 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.124790 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.124793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.124795 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:59.124852 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.124853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.125901 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.125903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.127046 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.127049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.128025 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.128026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.128088 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.128092 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.128122 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.128181 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":13,"spinlock_wait_cycles":896,"thread_start_us":67,"threads_started":1}
I20260808 02:12:59.128500 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.124193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:34081 (local address 127.0.0.1:56250)
0808 02:12:59.124327 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.124364 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.124423 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.124931 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.124932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.124934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:59.124968 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.124969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.126255 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.126256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.127278 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.127280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.128170 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.128171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.128221 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.128224 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.128252 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.128282 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":67,"threads_started":1}
I20260808 02:12:59.128556 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.124599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.19.58.195:34081)
0808 02:12:59.124762 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.124766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.124776 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:12:59.124809 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.124811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.124826 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.124929 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:59.125026 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.126170 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.127361 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.128086 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.128321 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.128389 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.128427 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.128464 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260808 02:12:59.128952 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.124459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46454 (local address 127.19.58.193:45537)
0808 02:12:59.124603 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.124608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.124619 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:12:59.124659 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.124661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.124677 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.124776 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:59.125058 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.125817 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.127140 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.127926 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.128712 (+ 786us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.128784 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.128809 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.128844 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260808 02:12:59.129134 22764 tablet_service.cc:1467] Processing CreateTablet for tablet b1143d35c169474f97404fad23e0d573 (DEFAULT_TABLE table=test-table [id=e30c637070434ee1a4b50d6771b009ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:59.129297 22994 tablet_service.cc:1467] Processing CreateTablet for tablet b1143d35c169474f97404fad23e0d573 (DEFAULT_TABLE table=test-table [id=e30c637070434ee1a4b50d6771b009ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:59.129505 22764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1143d35c169474f97404fad23e0d573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:59.129588 22994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1143d35c169474f97404fad23e0d573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:59.131841 23119 tablet_bootstrap.cc:492] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: Bootstrap starting.
I20260808 02:12:59.132192 23122 tablet_bootstrap.cc:492] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Bootstrap starting.
I20260808 02:12:59.132179 22554 tablet_service.cc:1467] Processing CreateTablet for tablet b1143d35c169474f97404fad23e0d573 (DEFAULT_TABLE table=test-table [id=e30c637070434ee1a4b50d6771b009ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:59.132498 22554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1143d35c169474f97404fad23e0d573. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:59.132606 23119 tablet_bootstrap.cc:654] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:59.132786 23122 tablet_bootstrap.cc:654] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:59.132828 23119 log.cc:824] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:59.133087 23122 log.cc:824] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:59.133489 23119 tablet_bootstrap.cc:492] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: No bootstrap required, opened a new log
I20260808 02:12:59.133545 23119 ts_tablet_manager.cc:1397] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:12:59.133801 23122 tablet_bootstrap.cc:492] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: No bootstrap required, opened a new log
I20260808 02:12:59.133857 23122 ts_tablet_manager.cc:1397] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:12:59.134789 23119 raft_consensus.cc:348] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } }
I20260808 02:12:59.134899 23119 raft_consensus.cc:374] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:59.134918 23119 raft_consensus.cc:729] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456336d0fb6841c8bd4b9e2ef0ecc607, State: Initialized, Role: FOLLOWER
I20260808 02:12:59.135003 23119 consensus_queue.cc:260] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } }
I20260808 02:12:59.135242 23119 ts_tablet_manager.cc:1428] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:59.135264 23094 heartbeater.cc:503] Master 127.19.58.254:44757 was elected leader, sending a full tablet report...
I20260808 02:12:59.135753 23122 raft_consensus.cc:348] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } }
I20260808 02:12:59.135897 23122 raft_consensus.cc:374] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:59.135933 23122 raft_consensus.cc:729] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5ee68c7a9844ba99e1c636fe0cd0f9, State: Initialized, Role: FOLLOWER
I20260808 02:12:59.136054 23122 consensus_queue.cc:260] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } }
I20260808 02:12:59.136312 23122 ts_tablet_manager.cc:1428] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:12:59.136335 22875 heartbeater.cc:503] Master 127.19.58.254:44757 was elected leader, sending a full tablet report...
I20260808 02:12:59.136499 23130 tablet_bootstrap.cc:492] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: Bootstrap starting.
I20260808 02:12:59.137018 23130 tablet_bootstrap.cc:654] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:59.137307 23130 log.cc:824] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:12:59.137897 23130 tablet_bootstrap.cc:492] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: No bootstrap required, opened a new log
I20260808 02:12:59.137948 23130 ts_tablet_manager.cc:1397] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:12:59.139187 23130 raft_consensus.cc:348] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } }
I20260808 02:12:59.139298 23130 raft_consensus.cc:374] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:59.139334 23130 raft_consensus.cc:729] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a449314c494146dc83c02a952a742c9c, State: Initialized, Role: FOLLOWER
I20260808 02:12:59.139451 23130 consensus_queue.cc:260] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } }
I20260808 02:12:59.139665 23130 ts_tablet_manager.cc:1428] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:12:59.139742 22654 heartbeater.cc:503] Master 127.19.58.254:44757 was elected leader, sending a full tablet report...
W20260808 02:12:59.196293 22876 tablet.cc:2369] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:12:59.196365 23131 raft_consensus.cc:482] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:59.196488 23131 raft_consensus.cc:504] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } }
I20260808 02:12:59.196818 23131 leader_election.cc:288] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081), a449314c494146dc83c02a952a742c9c (127.19.58.193:45537)
I20260808 02:12:59.200451 23135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.196999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:34081 (local address 127.19.58.194:33035)
0808 02:12:59.197194 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.197201 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.197246 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.197357 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.197358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.197360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:59.197432 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.197433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.198265 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.198266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.199546 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.199550 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.200195 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.200196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.200270 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.200273 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.200316 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.200359 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":56,"thread_start_us":124,"threads_started":1}
I20260808 02:12:59.200489 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.197083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:40577 (local address 127.19.58.193:45537)
0808 02:12:59.197124 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.197126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.197134 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:12:59.197165 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.197166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.197173 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.197231 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:59.197389 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.198142 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.199671 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.200142 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.200213 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.200222 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.200386 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.200429 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:12:59.200489 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.197061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:33035 (local address 127.19.58.195:34081)
0808 02:12:59.197115 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.197117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.197254 (+ 137us) server_negotiation.cc:407] Connection header received
0808 02:12:59.197286 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.197287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.197295 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.197340 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:59.197495 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.198198 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.199627 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.200116 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.200177 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.200186 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.200383 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.200426 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 02:12:59.200527 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.196996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45537 (local address 127.19.58.194:40577)
0808 02:12:59.197054 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.197066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.197112 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.197242 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.197243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.197244 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:59.197315 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.197317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.198232 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.198234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.199546 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.199550 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.200234 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.200235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.200313 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.200317 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.200374 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.200416 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":76}
I20260808 02:12:59.200842 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1143d35c169474f97404fad23e0d573" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a449314c494146dc83c02a952a742c9c" is_pre_election: true
I20260808 02:12:59.200935 23022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1143d35c169474f97404fad23e0d573" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" is_pre_election: true
I20260808 02:12:59.201028 22580 raft_consensus.cc:2454] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 0.
I20260808 02:12:59.201076 23022 raft_consensus.cc:2454] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 0.
I20260808 02:12:59.201252 22698 leader_election.cc:302] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a449314c494146dc83c02a952a742c9c, ee5ee68c7a9844ba99e1c636fe0cd0f9; no voters:
I20260808 02:12:59.201387 23131 raft_consensus.cc:2781] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:59.201447 23131 raft_consensus.cc:482] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:59.201462 23131 raft_consensus.cc:3037] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:59.202399 23131 raft_consensus.cc:504] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } }
I20260808 02:12:59.202515 23131 leader_election.cc:288] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 election: Requested vote from peers 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081), a449314c494146dc83c02a952a742c9c (127.19.58.193:45537)
I20260808 02:12:59.202669 23022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1143d35c169474f97404fad23e0d573" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607"
I20260808 02:12:59.202661 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1143d35c169474f97404fad23e0d573" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a449314c494146dc83c02a952a742c9c"
I20260808 02:12:59.202742 23022 raft_consensus.cc:3037] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:59.202745 22580 raft_consensus.cc:3037] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:59.203424 23022 raft_consensus.cc:2454] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 1.
I20260808 02:12:59.203593 22580 raft_consensus.cc:2454] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 1.
I20260808 02:12:59.203591 22691 leader_election.cc:302] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456336d0fb6841c8bd4b9e2ef0ecc607, ee5ee68c7a9844ba99e1c636fe0cd0f9; no voters:
I20260808 02:12:59.204272 23131 raft_consensus.cc:2781] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:59.204449 23131 raft_consensus.cc:686] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 LEADER]: Becoming Leader. State: Replica: ee5ee68c7a9844ba99e1c636fe0cd0f9, State: Running, Role: LEADER
I20260808 02:12:59.204528 23131 consensus_queue.cc:237] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } }
I20260808 02:12:59.205286 22337 catalog_manager.cc:5495] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194). New cstate: current_term: 1 leader_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } health_report { overall_health: HEALTHY } } }
Aug 08 02:12:59 dist-test-slave-1l3l krb5kdc[22254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155178, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:12:59.242487 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.236310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44757 (local address 127.0.0.1:45060)
0808 02:12:59.236453 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.236463 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.236521 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.236623 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.236623 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.236914 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:12:59.236958 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.236960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.237560 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.237561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.238084 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.238086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.238345 (+ 259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.238346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.238381 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.238876 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:12:59.238880 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:12:59.240272 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:12:59.241528 (+ 1256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:59.241536 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:59.241538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:59.241719 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:59.241884 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:12:59.241884 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:12:59.241885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:12:59.241936 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:12:59.242132 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:12:59.242141 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:12:59.242187 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.242247 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.242356 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260808 02:12:59.242494 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.236370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45060 (local address 127.19.58.254:44757)
0808 02:12:59.236455 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.236457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.236517 (+ 60us) server_negotiation.cc:407] Connection header received
0808 02:12:59.236558 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.236559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.236568 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.236613 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:12:59.237020 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.237506 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.238135 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.238302 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.238342 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.240363 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:12:59.240365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:12:59.240365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:12:59.240371 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:12:59.241456 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:59.241782 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:59.241783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:59.241783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:59.241831 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:12:59.241983 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:12:59.241984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:12:59.241985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:12:59.242081 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:12:59.242135 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.242275 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.242405 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":59}
I20260808 02:12:59.244467 22337 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:45060:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:12:59.244663 22337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:59.246058 22994 tablet_service.cc:1467] Processing CreateTablet for tablet 906cabd99c4546f0918e48ba40db4eb6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cccccb795b7f4d138d74d137c993bac6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:59.246083 22764 tablet_service.cc:1467] Processing CreateTablet for tablet 906cabd99c4546f0918e48ba40db4eb6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cccccb795b7f4d138d74d137c993bac6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:59.246078 22554 tablet_service.cc:1467] Processing CreateTablet for tablet 906cabd99c4546f0918e48ba40db4eb6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cccccb795b7f4d138d74d137c993bac6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:12:59.246222 22994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906cabd99c4546f0918e48ba40db4eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:59.246228 22764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906cabd99c4546f0918e48ba40db4eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:59.246229 22554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906cabd99c4546f0918e48ba40db4eb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:59.247334 23119 tablet_bootstrap.cc:492] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: Bootstrap starting.
I20260808 02:12:59.247772 23130 tablet_bootstrap.cc:492] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: Bootstrap starting.
I20260808 02:12:59.247852 23119 tablet_bootstrap.cc:654] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:59.248458 23130 tablet_bootstrap.cc:654] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:59.248586 23119 tablet_bootstrap.cc:492] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: No bootstrap required, opened a new log
I20260808 02:12:59.248627 23119 ts_tablet_manager.cc:1397] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:12:59.248764 23119 raft_consensus.cc:348] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.248811 23122 tablet_bootstrap.cc:492] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Bootstrap starting.
I20260808 02:12:59.248837 23119 raft_consensus.cc:374] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:59.248857 23119 raft_consensus.cc:729] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456336d0fb6841c8bd4b9e2ef0ecc607, State: Initialized, Role: FOLLOWER
I20260808 02:12:59.248911 23119 consensus_queue.cc:260] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.249017 23119 ts_tablet_manager.cc:1428] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:59.249063 23130 tablet_bootstrap.cc:492] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: No bootstrap required, opened a new log
I20260808 02:12:59.249105 23130 ts_tablet_manager.cc:1397] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:59.249074 23127 tablet_replica.cc:439] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } }
I20260808 02:12:59.249158 23127 tablet_replica.cc:442] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:59.249212 23122 tablet_bootstrap.cc:654] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:59.249249 23130 raft_consensus.cc:348] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.249325 23130 raft_consensus.cc:374] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:59.249344 23130 raft_consensus.cc:729] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a449314c494146dc83c02a952a742c9c, State: Initialized, Role: FOLLOWER
I20260808 02:12:59.249380 23130 consensus_queue.cc:260] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.249483 23130 ts_tablet_manager.cc:1428] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:59.249617 23133 tablet_replica.cc:439] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } }
I20260808 02:12:59.249713 23133 tablet_replica.cc:442] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:59.249964 23122 tablet_bootstrap.cc:492] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: No bootstrap required, opened a new log
I20260808 02:12:59.250013 23122 ts_tablet_manager.cc:1397] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:59.250137 23122 raft_consensus.cc:348] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.250195 23122 raft_consensus.cc:374] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:59.250213 23122 raft_consensus.cc:729] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5ee68c7a9844ba99e1c636fe0cd0f9, State: Initialized, Role: FOLLOWER
I20260808 02:12:59.250253 23122 consensus_queue.cc:260] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.250340 23122 ts_tablet_manager.cc:1428] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:59.250394 23131 tablet_replica.cc:439] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } }
I20260808 02:12:59.250484 23131 tablet_replica.cc:442] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 02:12:59.298659 22658 tablet.cc:2369] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:12:59.339803 23095 tablet.cc:2369] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:12:59.352365 23127 raft_consensus.cc:482] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:59.352444 23127 raft_consensus.cc:504] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.352753 23127 leader_election.cc:288] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a449314c494146dc83c02a952a742c9c (127.19.58.193:45537), ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194:38095)
I20260808 02:12:59.355362 22923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.352916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45537 (local address 127.19.58.195:40171)
0808 02:12:59.352964 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.352975 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.353022 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.353182 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.353183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.353185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:59.353252 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.353254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.353876 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.353877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.354602 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.354604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.355114 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.355115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.355195 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.355198 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.355243 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.355281 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 02:12:59.355372 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.353024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:40171 (local address 127.19.58.193:45537)
0808 02:12:59.353068 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.353069 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.353077 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:12:59.353104 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.353104 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.353112 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.353168 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:59.353312 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.353802 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.354692 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.355043 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.355104 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.355113 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.355259 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.355314 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:12:59.355481 23155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.352939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:38095 (local address 127.19.58.195:58291)
0808 02:12:59.353119 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.353126 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.353166 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.353383 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.353384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.353385 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:59.353446 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.353447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.354034 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.354035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.354736 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.354737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.355249 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.355250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.355319 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.355322 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.355367 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.355405 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":102,"threads_started":1}
I20260808 02:12:59.355480 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.353028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:58291 (local address 127.19.58.194:38095)
0808 02:12:59.353080 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.353082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.353176 (+ 94us) server_negotiation.cc:407] Connection header received
0808 02:12:59.353208 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.353209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.353217 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.353264 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:59.353517 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.353965 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.354814 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.355165 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.355224 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.355232 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.355386 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.355419 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 02:12:59.355623 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906cabd99c4546f0918e48ba40db4eb6" candidate_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a449314c494146dc83c02a952a742c9c" is_pre_election: true
I20260808 02:12:59.355728 22580 raft_consensus.cc:2454] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 456336d0fb6841c8bd4b9e2ef0ecc607 in term 0.
I20260808 02:12:59.355819 22792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906cabd99c4546f0918e48ba40db4eb6" candidate_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" is_pre_election: true
I20260808 02:12:59.355922 22792 raft_consensus.cc:2454] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 456336d0fb6841c8bd4b9e2ef0ecc607 in term 0.
I20260808 02:12:59.355934 22915 leader_election.cc:302] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456336d0fb6841c8bd4b9e2ef0ecc607, a449314c494146dc83c02a952a742c9c; no voters:
I20260808 02:12:59.356063 23127 raft_consensus.cc:2781] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:59.356118 23127 raft_consensus.cc:482] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:59.356144 23127 raft_consensus.cc:3037] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:59.356824 23127 raft_consensus.cc:504] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.356936 23127 leader_election.cc:288] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [CANDIDATE]: Term 1 election: Requested vote from peers a449314c494146dc83c02a952a742c9c (127.19.58.193:45537), ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194:38095)
I20260808 02:12:59.357093 22792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906cabd99c4546f0918e48ba40db4eb6" candidate_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9"
I20260808 02:12:59.357091 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906cabd99c4546f0918e48ba40db4eb6" candidate_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a449314c494146dc83c02a952a742c9c"
I20260808 02:12:59.357172 22792 raft_consensus.cc:3037] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:59.357170 22580 raft_consensus.cc:3037] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:59.357960 22580 raft_consensus.cc:2454] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 456336d0fb6841c8bd4b9e2ef0ecc607 in term 1.
I20260808 02:12:59.357960 22792 raft_consensus.cc:2454] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 456336d0fb6841c8bd4b9e2ef0ecc607 in term 1.
I20260808 02:12:59.358140 22910 leader_election.cc:302] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456336d0fb6841c8bd4b9e2ef0ecc607, ee5ee68c7a9844ba99e1c636fe0cd0f9; no voters:
I20260808 02:12:59.358227 23127 raft_consensus.cc:2781] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:59.358393 23127 raft_consensus.cc:686] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 LEADER]: Becoming Leader. State: Replica: 456336d0fb6841c8bd4b9e2ef0ecc607, State: Running, Role: LEADER
I20260808 02:12:59.358482 23127 consensus_queue.cc:237] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.358875 23156 tablet_replica.cc:439] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: TxnStatusTablet state changed. Reason: New leader 456336d0fb6841c8bd4b9e2ef0ecc607. Latest consensus state: current_term: 1 leader_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } }
I20260808 02:12:59.358959 23156 tablet_replica.cc:442] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:59.359127 23159 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:12:59.359241 23159 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:12:59.359223 22337 catalog_manager.cc:5495] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 reported cstate change: term changed from 0 to 1, leader changed from <none> to 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195). New cstate: current_term: 1 leader_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } health_report { overall_health: HEALTHY } } }
I20260808 02:12:59.366016 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.363827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56264 (local address 127.19.58.195:34081)
0808 02:12:59.363879 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.363881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.363888 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:12:59.363996 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.363997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.364014 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.364070 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:59.364187 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.364626 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.365320 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.365516 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.365568 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.365733 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.365820 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.365887 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.365937 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.365957 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:12:59.366043 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.363773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:34081 (local address 127.0.0.1:56264)
0808 02:12:59.363847 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.363857 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.363925 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.364081 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.364082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.364083 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:59.364132 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.364135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.364694 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.364695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.365251 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.365253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.365574 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.365575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.365644 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.365658 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.365876 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.365884 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.365934 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.365978 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 02:12:59.367494 22792 raft_consensus.cc:1261] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Refusing update from remote peer 456336d0fb6841c8bd4b9e2ef0ecc607: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:59.367570 22580 raft_consensus.cc:1261] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Refusing update from remote peer 456336d0fb6841c8bd4b9e2ef0ecc607: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:59.367798 23127 consensus_queue.cc:1035] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:59.367965 23156 consensus_queue.cc:1035] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:59.368947 23131 tablet_replica.cc:439] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: TxnStatusTablet state changed. Reason: New leader 456336d0fb6841c8bd4b9e2ef0ecc607. Latest consensus state: current_term: 1 leader_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } }
I20260808 02:12:59.369033 23131 tablet_replica.cc:442] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:59.368999 23133 tablet_replica.cc:439] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: TxnStatusTablet state changed. Reason: New leader 456336d0fb6841c8bd4b9e2ef0ecc607. Latest consensus state: current_term: 1 leader_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } }
I20260808 02:12:59.369079 23133 tablet_replica.cc:442] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:59.369930 23156 tablet_replica.cc:439] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } }
I20260808 02:12:59.370011 23156 tablet_replica.cc:442] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:59.370370 23127 tablet_replica.cc:439] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } }
I20260808 02:12:59.370437 23127 tablet_replica.cc:442] T 906cabd99c4546f0918e48ba40db4eb6 P 456336d0fb6841c8bd4b9e2ef0ecc607: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:12:59.370522 23131 tablet_replica.cc:439] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } }
I20260808 02:12:59.370601 23131 tablet_replica.cc:442] T 906cabd99c4546f0918e48ba40db4eb6 P ee5ee68c7a9844ba99e1c636fe0cd0f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:59.371260 23133 tablet_replica.cc:439] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } } }
I20260808 02:12:59.371337 23133 tablet_replica.cc:442] T 906cabd99c4546f0918e48ba40db4eb6 P a449314c494146dc83c02a952a742c9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:12:59.371742 23165 mvcc.cc:205] Tried to move back new op lower bound from 7316091614687240192 to 7316091614652854272. Current Snapshot: MvccSnapshot[applied={T|T < 7316091614687240192}]
I20260808 02:12:59.372085 23164 mvcc.cc:205] Tried to move back new op lower bound from 7316091614687240192 to 7316091614652854272. Current Snapshot: MvccSnapshot[applied={T|T < 7316091614687240192 or (T in {7316091614687240192})}]
I20260808 02:12:59.374720 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.372028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.19.58.194:38095)
0808 02:12:59.372176 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.372178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.372185 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:12:59.372211 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.372211 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.372227 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.372284 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:59.372466 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.372991 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.374099 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.374290 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.374339 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.374484 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.374563 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.374613 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.374660 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.374675 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260808 02:12:59.379566 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.376948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:48427 (local address 127.19.58.195:34081)
0808 02:12:59.377000 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.377002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.377010 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:12:59.377070 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.377070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.377081 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.377120 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:59.377233 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.378242 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.378934 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.379133 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.379183 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.379346 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.379377 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.379425 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.379490 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.379507 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 02:12:59.379585 22881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.376887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:34081 (local address 127.19.58.194:48427)
0808 02:12:59.376957 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.376968 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.377006 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.377132 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.377133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.377134 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:59.377185 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.377186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.378327 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.378328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.378865 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.378867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.379197 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.379198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.379259 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.379272 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.379438 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.379440 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.379486 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.379526 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 02:12:59.381753 23022 raft_consensus.cc:1261] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Refusing update from remote peer ee5ee68c7a9844ba99e1c636fe0cd0f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:59.382014 23136 consensus_queue.cc:1035] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:59.382038 22580 raft_consensus.cc:1261] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Refusing update from remote peer ee5ee68c7a9844ba99e1c636fe0cd0f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:59.382228 23136 consensus_queue.cc:1035] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:59.390093 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.387722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:49597 (local address 127.19.58.194:38095)
0808 02:12:59.387755 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.387756 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.387802 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:12:59.387870 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.387870 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.387877 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.387925 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:59.388035 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.388458 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.389544 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.389716 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.389761 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.389932 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.389953 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.389982 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.390032 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.390046 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:12:59.390142 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.387664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:38095 (local address 127.19.58.195:49597)
0808 02:12:59.387758 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.387768 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.387819 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.387929 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.387930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.387931 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:59.387985 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.387986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.388519 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.388519 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.389472 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.389474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.389775 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.389776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.389839 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.389853 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.389985 (+ 132us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.389986 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.390030 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.390073 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 02:12:59.399550 22337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:59.399650 22337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:12:59.400403 22337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86: Sending DeleteTablet for 3 replicas of tablet b1143d35c169474f97404fad23e0d573
I20260808 02:12:59.400695 22554 tablet_service.cc:1514] Processing DeleteTablet for tablet b1143d35c169474f97404fad23e0d573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:59 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:46454
I20260808 02:12:59.400765 22994 tablet_service.cc:1514] Processing DeleteTablet for tablet b1143d35c169474f97404fad23e0d573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:59 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:56250
I20260808 02:12:59.400991 22764 tablet_service.cc:1514] Processing DeleteTablet for tablet b1143d35c169474f97404fad23e0d573 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:12:59 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:47792
I20260808 02:12:59.401201 23186 tablet_replica.cc:330] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: stopping tablet replica
I20260808 02:12:59.401224 22337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:12:59.401360 22337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:12:59.401523 23186 raft_consensus.cc:2229] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:12:59.401672 23186 raft_consensus.cc:2258] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:59.402174 23186 ts_tablet_manager.cc:1905] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:59.402354 22554 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4a7fc7dece4898a02d41d358d12f94 (DEFAULT_TABLE table=test-table [id=91f7160238594765b25d766302702ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:59.402513 22554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4a7fc7dece4898a02d41d358d12f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:59.402580 22994 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4a7fc7dece4898a02d41d358d12f94 (DEFAULT_TABLE table=test-table [id=91f7160238594765b25d766302702ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:59.402678 22994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4a7fc7dece4898a02d41d358d12f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:59.402717 22764 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4a7fc7dece4898a02d41d358d12f94 (DEFAULT_TABLE table=test-table [id=91f7160238594765b25d766302702ccc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:12:59.402807 22764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4a7fc7dece4898a02d41d358d12f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:12:59.403861 23130 tablet_bootstrap.cc:492] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c: Bootstrap starting.
I20260808 02:12:59.403945 23119 tablet_bootstrap.cc:492] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607: Bootstrap starting.
I20260808 02:12:59.404062 23122 tablet_bootstrap.cc:492] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Bootstrap starting.
I20260808 02:12:59.404243 23130 tablet_bootstrap.cc:654] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:59.404349 23119 tablet_bootstrap.cc:654] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:59.404423 23122 tablet_bootstrap.cc:654] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Neither blocks nor log segments found. Creating new log.
I20260808 02:12:59.405251 23122 tablet_bootstrap.cc:492] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9: No bootstrap required, opened a new log
I20260808 02:12:59.405304 23122 ts_tablet_manager.cc:1397] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:12:59.405449 23122 raft_consensus.cc:348] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.405515 23122 raft_consensus.cc:374] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:59.405541 23122 raft_consensus.cc:729] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5ee68c7a9844ba99e1c636fe0cd0f9, State: Initialized, Role: FOLLOWER
I20260808 02:12:59.405579 23122 consensus_queue.cc:260] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.405669 23122 ts_tablet_manager.cc:1428] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:59.405797 23184 tablet_replica.cc:330] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: stopping tablet replica
I20260808 02:12:59.405885 23184 raft_consensus.cc:2229] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:59.405948 23184 raft_consensus.cc:2258] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:59.406622 23184 ts_tablet_manager.cc:1905] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:59.407598 23119 tablet_bootstrap.cc:492] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607: No bootstrap required, opened a new log
I20260808 02:12:59.407646 23119 ts_tablet_manager.cc:1397] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 02:12:59.407784 23119 raft_consensus.cc:348] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.407846 23119 raft_consensus.cc:374] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:59.407866 23119 raft_consensus.cc:729] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456336d0fb6841c8bd4b9e2ef0ecc607, State: Initialized, Role: FOLLOWER
I20260808 02:12:59.407914 23119 consensus_queue.cc:260] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.407997 23119 ts_tablet_manager.cc:1428] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:12:59.408447 23184 ts_tablet_manager.cc:1918] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:59.408519 23184 log.cc:1196] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/b1143d35c169474f97404fad23e0d573
I20260808 02:12:59.408787 23184 ts_tablet_manager.cc:1939] T b1143d35c169474f97404fad23e0d573 P a449314c494146dc83c02a952a742c9c: Deleting consensus metadata
I20260808 02:12:59.409190 22303 catalog_manager.cc:4841] TS a449314c494146dc83c02a952a742c9c (127.19.58.193:45537): tablet b1143d35c169474f97404fad23e0d573 (table test-table [id=e30c637070434ee1a4b50d6771b009ad]) successfully deleted
I20260808 02:12:59.411813 23186 ts_tablet_manager.cc:1918] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:59.411887 23186 log.cc:1196] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/b1143d35c169474f97404fad23e0d573
I20260808 02:12:59.412073 23130 tablet_bootstrap.cc:492] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c: No bootstrap required, opened a new log
I20260808 02:12:59.412127 23130 ts_tablet_manager.cc:1397] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260808 02:12:59.412142 23186 ts_tablet_manager.cc:1939] T b1143d35c169474f97404fad23e0d573 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting consensus metadata
I20260808 02:12:59.412269 23130 raft_consensus.cc:348] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.412339 23130 raft_consensus.cc:374] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:12:59.412380 23130 raft_consensus.cc:729] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a449314c494146dc83c02a952a742c9c, State: Initialized, Role: FOLLOWER
I20260808 02:12:59.412421 23130 consensus_queue.cc:260] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.412508 23130 ts_tablet_manager.cc:1428] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 02:12:59.412667 22299 catalog_manager.cc:4841] TS ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194:38095): tablet b1143d35c169474f97404fad23e0d573 (table test-table [id=e30c637070434ee1a4b50d6771b009ad]) successfully deleted
I20260808 02:12:59.412943 23185 tablet_replica.cc:330] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: stopping tablet replica
I20260808 02:12:59.413051 23185 raft_consensus.cc:2229] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:12:59.413112 23185 raft_consensus.cc:2258] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:12:59.413516 23185 ts_tablet_manager.cc:1905] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:12:59.415529 23185 ts_tablet_manager.cc:1918] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:12:59.415599 23185 log.cc:1196] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/b1143d35c169474f97404fad23e0d573
I20260808 02:12:59.415863 23185 ts_tablet_manager.cc:1939] T b1143d35c169474f97404fad23e0d573 P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting consensus metadata
I20260808 02:12:59.416472 22299 catalog_manager.cc:4841] TS 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081): tablet b1143d35c169474f97404fad23e0d573 (table test-table [id=e30c637070434ee1a4b50d6771b009ad]) successfully deleted
I20260808 02:12:59.489588 23133 raft_consensus.cc:482] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:12:59.489709 23133 raft_consensus.cc:504] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.490056 23133 leader_election.cc:288] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194:38095), 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081)
I20260808 02:12:59.494038 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.490536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:54425 (local address 127.19.58.195:34081)
0808 02:12:59.490579 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.490581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.490718 (+ 137us) server_negotiation.cc:407] Connection header received
0808 02:12:59.490855 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.490856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.490864 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.490898 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:59.491086 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.491909 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.493224 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.493676 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.493721 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.493730 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.493937 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.493973 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:12:59.494175 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.490319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:36513 (local address 127.19.58.194:38095)
0808 02:12:59.490367 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.490369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.490594 (+ 225us) server_negotiation.cc:407] Connection header received
0808 02:12:59.490700 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.490701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.490712 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.490748 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:12:59.491002 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.491765 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.493262 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.493779 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.493827 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.493836 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.494073 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.494107 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:12:59.494712 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.490251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:38095 (local address 127.19.58.193:36513)
0808 02:12:59.490517 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.490529 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.490625 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.490820 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.490820 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.490822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:59.490913 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.490915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.491909 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.491910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.493155 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.493158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.493910 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.493911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.493990 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.493994 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.494576 (+ 582us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.494623 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":7,"thread_start_us":66,"threads_started":1}
I20260808 02:12:59.495101 22792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4a7fc7dece4898a02d41d358d12f94" candidate_uuid: "a449314c494146dc83c02a952a742c9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" is_pre_election: true
I20260808 02:12:59.495194 22792 raft_consensus.cc:2454] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a449314c494146dc83c02a952a742c9c in term 0.
I20260808 02:12:59.495487 22480 leader_election.cc:302] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a449314c494146dc83c02a952a742c9c, ee5ee68c7a9844ba99e1c636fe0cd0f9; no voters:
I20260808 02:12:59.495609 23133 raft_consensus.cc:2781] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:12:59.495636 23133 raft_consensus.cc:482] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:12:59.495654 23133 raft_consensus.cc:3037] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:59.496572 23133 raft_consensus.cc:504] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.496702 23133 leader_election.cc:288] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [CANDIDATE]: Term 1 election: Requested vote from peers ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194:38095), 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081)
I20260808 02:12:59.496918 22792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4a7fc7dece4898a02d41d358d12f94" candidate_uuid: "a449314c494146dc83c02a952a742c9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9"
I20260808 02:12:59.496997 22792 raft_consensus.cc:3037] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:59.497757 22792 raft_consensus.cc:2454] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a449314c494146dc83c02a952a742c9c in term 1.
I20260808 02:12:59.497915 22480 leader_election.cc:302] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a449314c494146dc83c02a952a742c9c, ee5ee68c7a9844ba99e1c636fe0cd0f9; no voters:
I20260808 02:12:59.498034 23133 raft_consensus.cc:2781] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:12:59.498165 23133 raft_consensus.cc:686] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 1 LEADER]: Becoming Leader. State: Replica: a449314c494146dc83c02a952a742c9c, State: Running, Role: LEADER
I20260808 02:12:59.498216 23133 consensus_queue.cc:237] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:12:59.498836 22338 catalog_manager.cc:5495] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c reported cstate change: term changed from 0 to 1, leader changed from <none> to a449314c494146dc83c02a952a742c9c (127.19.58.193). New cstate: current_term: 1 leader_uuid: "a449314c494146dc83c02a952a742c9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } health_report { overall_health: UNKNOWN } } }
I20260808 02:12:59.503980 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.490428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:34081 (local address 127.19.58.193:54425)
0808 02:12:59.490664 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.490673 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.490741 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.490955 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.490956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.490957 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:12:59.491026 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.491027 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.492036 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.492037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.493124 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.493127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.493784 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.493785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.493853 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.493857 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.494154 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.503876 (+ 9722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":29,"threads_started":1}
I20260808 02:12:59.504242 23022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4a7fc7dece4898a02d41d358d12f94" candidate_uuid: "a449314c494146dc83c02a952a742c9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" is_pre_election: true
I20260808 02:12:59.504339 23022 raft_consensus.cc:2454] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a449314c494146dc83c02a952a742c9c in term 0.
I20260808 02:12:59.504278 23020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4a7fc7dece4898a02d41d358d12f94" candidate_uuid: "a449314c494146dc83c02a952a742c9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607"
I20260808 02:12:59.504374 23020 raft_consensus.cc:3037] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:12:59.505165 23020 raft_consensus.cc:2454] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a449314c494146dc83c02a952a742c9c in term 1.
I20260808 02:12:59.525022 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.522713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.19.58.193:45537)
0808 02:12:59.522760 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.522762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.522770 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:12:59.522831 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.522832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.522851 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.522912 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:59.523042 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.523528 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.524283 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.524482 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.524537 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.524680 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.524807 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.524871 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.524933 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.524953 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:12:59.529170 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.527217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:41297 (local address 127.19.58.195:34081)
0808 02:12:59.527264 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.527266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.527346 (+ 80us) server_negotiation.cc:407] Connection header received
0808 02:12:59.527386 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.527387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.527396 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.527442 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:59.527555 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.528012 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.528689 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.528858 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.528906 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.529024 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.529047 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.529073 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.529107 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.529121 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:12:59.529207 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.527139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:34081 (local address 127.19.58.193:41297)
0808 02:12:59.527291 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.527301 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.527351 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.527454 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.527455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.527456 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:59.527516 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.527517 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.528064 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.528064 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.528629 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.528631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.528917 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.528917 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.528957 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.528970 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.529078 (+ 108us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.529079 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.529105 (+ 26us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.529135 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260808 02:12:59.531157 22792 raft_consensus.cc:1261] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Refusing update from remote peer a449314c494146dc83c02a952a742c9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:59.531198 23020 raft_consensus.cc:1261] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Refusing update from remote peer a449314c494146dc83c02a952a742c9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:12:59.531430 23195 consensus_queue.cc:1035] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:59.531550 23133 consensus_queue.cc:1035] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:12:59.532760 23163 mvcc.cc:205] Tried to move back new op lower bound from 7316091615358156800 to 7316091615225106432. Current Snapshot: MvccSnapshot[applied={T|T < 7316091615358156800}]
I20260808 02:12:59.537909 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.535944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:32909 (local address 127.19.58.193:45537)
0808 02:12:59.535981 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 02:12:59.535982 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:12:59.535990 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:12:59.536037 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:12:59.536038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:12:59.536047 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:12:59.536099 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:12:59.536190 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.536635 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.537288 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.537490 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.537542 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.537711 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.537738 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.537786 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 02:12:59.537845 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 02:12:59.537863 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:12:59.537946 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:12:59.535907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45537 (local address 127.19.58.195:32909)
0808 02:12:59.535952 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 02:12:59.535961 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:12:59.535993 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:12:59.536100 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:12:59.536101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:12:59.536102 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:12:59.536141 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.536142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.536700 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.536701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.537223 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:12:59.537224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:12:59.537547 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:12:59.537549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:12:59.537603 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:12:59.537607 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:12:59.537789 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:12:59.537791 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:12:59.537835 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:12:59.537881 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 02:12:59.538249 23165 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 02:13:01.056923 22335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:01.057031 22335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:01.057782 22335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86: Sending DeleteTablet for 3 replicas of tablet 8a4a7fc7dece4898a02d41d358d12f94
I20260808 02:13:01.058003 22554 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4a7fc7dece4898a02d41d358d12f94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:01 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:46454
I20260808 02:13:01.058073 22764 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4a7fc7dece4898a02d41d358d12f94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:01 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:47792
I20260808 02:13:01.058113 22994 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4a7fc7dece4898a02d41d358d12f94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:01 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:56250
I20260808 02:13:01.058198 23242 tablet_replica.cc:330] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c: stopping tablet replica
I20260808 02:13:01.058245 23242 raft_consensus.cc:2229] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:01.058324 23242 raft_consensus.cc:2258] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:01.058565 23242 ts_tablet_manager.cc:1905] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:01.059870 23242 ts_tablet_manager.cc:1918] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:13:01.059938 23242 log.cc:1196] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/8a4a7fc7dece4898a02d41d358d12f94
I20260808 02:13:01.060199 23242 ts_tablet_manager.cc:1939] T 8a4a7fc7dece4898a02d41d358d12f94 P a449314c494146dc83c02a952a742c9c: Deleting consensus metadata
I20260808 02:13:01.060506 22303 catalog_manager.cc:4841] TS a449314c494146dc83c02a952a742c9c (127.19.58.193:45537): tablet 8a4a7fc7dece4898a02d41d358d12f94 (table test-table [id=91f7160238594765b25d766302702ccc]) successfully deleted
I20260808 02:13:01.060650 23245 tablet_replica.cc:330] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607: stopping tablet replica
I20260808 02:13:01.060712 23245 raft_consensus.cc:2229] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:01.060750 23245 raft_consensus.cc:2258] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:01.060850 23243 tablet_replica.cc:330] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9: stopping tablet replica
I20260808 02:13:01.060904 23243 raft_consensus.cc:2229] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:01.060945 23243 raft_consensus.cc:2258] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:01.061129 23245 ts_tablet_manager.cc:1905] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:01.062251 23245 ts_tablet_manager.cc:1918] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:13:01.062299 23245 log.cc:1196] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/8a4a7fc7dece4898a02d41d358d12f94
I20260808 02:13:01.062453 23245 ts_tablet_manager.cc:1939] T 8a4a7fc7dece4898a02d41d358d12f94 P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting consensus metadata
I20260808 02:13:01.062878 22299 catalog_manager.cc:4841] TS 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081): tablet 8a4a7fc7dece4898a02d41d358d12f94 (table test-table [id=91f7160238594765b25d766302702ccc]) successfully deleted
I20260808 02:13:01.063113 23243 ts_tablet_manager.cc:1905] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:01.064729 23243 ts_tablet_manager.cc:1918] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:13:01.064793 23243 log.cc:1196] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/8a4a7fc7dece4898a02d41d358d12f94
I20260808 02:13:01.065004 23243 ts_tablet_manager.cc:1939] T 8a4a7fc7dece4898a02d41d358d12f94 P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting consensus metadata
I20260808 02:13:01.065302 22299 catalog_manager.cc:4841] TS ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194:38095): tablet 8a4a7fc7dece4898a02d41d358d12f94 (table test-table [id=91f7160238594765b25d766302702ccc]) successfully deleted
Aug 08 02:13:01 dist-test-slave-1l3l krb5kdc[22254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155181, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:13:01 dist-test-slave-1l3l krb5kdc[22254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155181, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:01.085427 23254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:01.076149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.19.58.254:44757)
0808 02:13:01.076269 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:01.076271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:01.076283 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:13:01.076344 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:01.076344 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:01.076354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:01.076436 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:01.076935 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:01.077446 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:01.078090 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:01.078276 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:01.078317 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:01.083346 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:01.083348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:01.083348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:01.083354 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:01.084583 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:01.084893 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:01.084894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:01.084894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:01.084941 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:01.085072 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:01.085073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:01.085073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:01.085165 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:01.085231 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:01.085302 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 02:13:01.085360 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260808 02:13:01.086722 22335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:01.086858 22335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:01.088155 22764 tablet_service.cc:1467] Processing CreateTablet for tablet 40fbb2cfcba647358cf54739abd3949f (DEFAULT_TABLE table=test-table [id=379ca05882814da8b607368fe7f7774d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:01.088155 22554 tablet_service.cc:1467] Processing CreateTablet for tablet 40fbb2cfcba647358cf54739abd3949f (DEFAULT_TABLE table=test-table [id=379ca05882814da8b607368fe7f7774d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:01.088168 22994 tablet_service.cc:1467] Processing CreateTablet for tablet 40fbb2cfcba647358cf54739abd3949f (DEFAULT_TABLE table=test-table [id=379ca05882814da8b607368fe7f7774d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:01.088287 22994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40fbb2cfcba647358cf54739abd3949f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:01.088289 22764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40fbb2cfcba647358cf54739abd3949f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:01.088296 22554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40fbb2cfcba647358cf54739abd3949f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:01.089754 23257 tablet_bootstrap.cc:492] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607: Bootstrap starting.
I20260808 02:13:01.090145 23257 tablet_bootstrap.cc:654] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:01.090446 23258 tablet_bootstrap.cc:492] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9: Bootstrap starting.
I20260808 02:13:01.090741 23257 tablet_bootstrap.cc:492] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607: No bootstrap required, opened a new log
I20260808 02:13:01.090798 23257 ts_tablet_manager.cc:1397] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:01.090839 23258 tablet_bootstrap.cc:654] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:01.090929 23257 raft_consensus.cc:348] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.091001 23257 raft_consensus.cc:374] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:01.091059 23257 raft_consensus.cc:729] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456336d0fb6841c8bd4b9e2ef0ecc607, State: Initialized, Role: FOLLOWER
I20260808 02:13:01.091100 23256 tablet_bootstrap.cc:492] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c: Bootstrap starting.
I20260808 02:13:01.091125 23257 consensus_queue.cc:260] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.091338 23257 ts_tablet_manager.cc:1428] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:13:01.091465 23256 tablet_bootstrap.cc:654] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:01.091464 23258 tablet_bootstrap.cc:492] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9: No bootstrap required, opened a new log
I20260808 02:13:01.091524 23258 ts_tablet_manager.cc:1397] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:01.091634 23258 raft_consensus.cc:348] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.091739 23258 raft_consensus.cc:374] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:01.091771 23258 raft_consensus.cc:729] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5ee68c7a9844ba99e1c636fe0cd0f9, State: Initialized, Role: FOLLOWER
I20260808 02:13:01.091813 23258 consensus_queue.cc:260] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.091975 23258 ts_tablet_manager.cc:1428] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:01.092587 23256 tablet_bootstrap.cc:492] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c: No bootstrap required, opened a new log
I20260808 02:13:01.092640 23256 ts_tablet_manager.cc:1397] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 02:13:01.092734 23256 raft_consensus.cc:348] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.092809 23256 raft_consensus.cc:374] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:01.092837 23256 raft_consensus.cc:729] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a449314c494146dc83c02a952a742c9c, State: Initialized, Role: FOLLOWER
I20260808 02:13:01.092911 23256 consensus_queue.cc:260] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.093063 23256 ts_tablet_manager.cc:1428] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:01.345914 23264 raft_consensus.cc:482] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:01.346024 23264 raft_consensus.cc:504] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.346194 23264 leader_election.cc:288] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a449314c494146dc83c02a952a742c9c (127.19.58.193:45537), 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081)
I20260808 02:13:01.346418 23020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40fbb2cfcba647358cf54739abd3949f" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" is_pre_election: true
I20260808 02:13:01.346436 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40fbb2cfcba647358cf54739abd3949f" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a449314c494146dc83c02a952a742c9c" is_pre_election: true
I20260808 02:13:01.346516 23020 raft_consensus.cc:2454] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 0.
I20260808 02:13:01.346537 22580 raft_consensus.cc:2454] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 0.
I20260808 02:13:01.346698 22691 leader_election.cc:302] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456336d0fb6841c8bd4b9e2ef0ecc607, ee5ee68c7a9844ba99e1c636fe0cd0f9; no voters:
I20260808 02:13:01.346818 23264 raft_consensus.cc:2781] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:01.346873 23264 raft_consensus.cc:482] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:01.346892 23264 raft_consensus.cc:3037] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:01.347597 23264 raft_consensus.cc:504] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.347738 23264 leader_election.cc:288] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 election: Requested vote from peers a449314c494146dc83c02a952a742c9c (127.19.58.193:45537), 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081)
I20260808 02:13:01.347867 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40fbb2cfcba647358cf54739abd3949f" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a449314c494146dc83c02a952a742c9c"
I20260808 02:13:01.347893 23020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40fbb2cfcba647358cf54739abd3949f" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607"
I20260808 02:13:01.347940 22580 raft_consensus.cc:3037] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:01.347959 23020 raft_consensus.cc:3037] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:01.348634 23020 raft_consensus.cc:2454] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 1.
I20260808 02:13:01.348672 22580 raft_consensus.cc:2454] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 1.
I20260808 02:13:01.348838 22698 leader_election.cc:302] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a449314c494146dc83c02a952a742c9c, ee5ee68c7a9844ba99e1c636fe0cd0f9; no voters:
I20260808 02:13:01.348973 23264 raft_consensus.cc:2781] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:01.349161 23264 raft_consensus.cc:686] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 LEADER]: Becoming Leader. State: Replica: ee5ee68c7a9844ba99e1c636fe0cd0f9, State: Running, Role: LEADER
I20260808 02:13:01.349215 23264 consensus_queue.cc:237] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.349790 22335 catalog_manager.cc:5495] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194). New cstate: current_term: 1 leader_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:01.374936 23270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:01.372640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.19.58.194:38095)
0808 02:13:01.372800 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:01.372802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:01.372808 (+ 6us) server_negotiation.cc:407] Connection header received
0808 02:13:01.372825 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:01.372826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:01.372835 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:01.372903 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:01.373051 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:01.373556 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:01.374288 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:01.374502 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:01.374557 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:01.374712 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:01.374748 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:01.374788 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:01.374837 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 02:13:01.374858 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260808 02:13:01.376952 22580 raft_consensus.cc:1261] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Refusing update from remote peer ee5ee68c7a9844ba99e1c636fe0cd0f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:01.376950 23020 raft_consensus.cc:1261] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Refusing update from remote peer ee5ee68c7a9844ba99e1c636fe0cd0f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:01.377135 23265 consensus_queue.cc:1035] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:01.377238 23264 consensus_queue.cc:1035] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:01.386458 22337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:01.386536 22337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:01.387086 22337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86: Sending DeleteTablet for 3 replicas of tablet 40fbb2cfcba647358cf54739abd3949f
I20260808 02:13:01.387288 22554 tablet_service.cc:1514] Processing DeleteTablet for tablet 40fbb2cfcba647358cf54739abd3949f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:01 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:46454
I20260808 02:13:01.387312 22764 tablet_service.cc:1514] Processing DeleteTablet for tablet 40fbb2cfcba647358cf54739abd3949f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:01 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:47792
I20260808 02:13:01.387375 23242 tablet_replica.cc:330] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c: stopping tablet replica
I20260808 02:13:01.387351 22994 tablet_service.cc:1514] Processing DeleteTablet for tablet 40fbb2cfcba647358cf54739abd3949f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:01 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:56250
I20260808 02:13:01.387418 23242 raft_consensus.cc:2229] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:01.387447 23243 tablet_replica.cc:330] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9: stopping tablet replica
I20260808 02:13:01.387466 23242 raft_consensus.cc:2258] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:01.387506 23243 raft_consensus.cc:2229] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:01.387539 23245 tablet_replica.cc:330] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607: stopping tablet replica
I20260808 02:13:01.387558 23243 raft_consensus.cc:2258] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:01.387601 23245 raft_consensus.cc:2229] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:01.387636 23245 raft_consensus.cc:2258] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:01.387631 22337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:01.387780 22337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:01.387917 23243 ts_tablet_manager.cc:1905] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:01.387988 23242 ts_tablet_manager.cc:1905] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:01.388063 23245 ts_tablet_manager.cc:1905] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:01.389308 23243 ts_tablet_manager.cc:1918] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:01.389371 23243 log.cc:1196] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/40fbb2cfcba647358cf54739abd3949f
I20260808 02:13:01.389524 23242 ts_tablet_manager.cc:1918] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:01.389544 23245 ts_tablet_manager.cc:1918] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:01.389577 23242 log.cc:1196] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/40fbb2cfcba647358cf54739abd3949f
I20260808 02:13:01.389595 23245 log.cc:1196] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/40fbb2cfcba647358cf54739abd3949f
I20260808 02:13:01.389652 23243 ts_tablet_manager.cc:1939] T 40fbb2cfcba647358cf54739abd3949f P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting consensus metadata
I20260808 02:13:01.389770 23242 ts_tablet_manager.cc:1939] T 40fbb2cfcba647358cf54739abd3949f P a449314c494146dc83c02a952a742c9c: Deleting consensus metadata
I20260808 02:13:01.389811 23245 ts_tablet_manager.cc:1939] T 40fbb2cfcba647358cf54739abd3949f P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting consensus metadata
I20260808 02:13:01.390020 22299 catalog_manager.cc:4841] TS ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194:38095): tablet 40fbb2cfcba647358cf54739abd3949f (table test-table [id=379ca05882814da8b607368fe7f7774d]) successfully deleted
I20260808 02:13:01.390213 22303 catalog_manager.cc:4841] TS a449314c494146dc83c02a952a742c9c (127.19.58.193:45537): tablet 40fbb2cfcba647358cf54739abd3949f (table test-table [id=379ca05882814da8b607368fe7f7774d]) successfully deleted
I20260808 02:13:01.390261 22299 catalog_manager.cc:4841] TS 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081): tablet 40fbb2cfcba647358cf54739abd3949f (table test-table [id=379ca05882814da8b607368fe7f7774d]) successfully deleted
I20260808 02:13:01.390407 22554 tablet_service.cc:1467] Processing CreateTablet for tablet e0f05df22f2a4181b13b88f9c4bc9b9a (DEFAULT_TABLE table=test-table [id=27f3c442c7384121b8368771d8428112]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:01.390466 22994 tablet_service.cc:1467] Processing CreateTablet for tablet e0f05df22f2a4181b13b88f9c4bc9b9a (DEFAULT_TABLE table=test-table [id=27f3c442c7384121b8368771d8428112]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:01.390487 22764 tablet_service.cc:1467] Processing CreateTablet for tablet e0f05df22f2a4181b13b88f9c4bc9b9a (DEFAULT_TABLE table=test-table [id=27f3c442c7384121b8368771d8428112]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:01.390522 22554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f05df22f2a4181b13b88f9c4bc9b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:01.390571 22764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f05df22f2a4181b13b88f9c4bc9b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:01.390573 22994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f05df22f2a4181b13b88f9c4bc9b9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:01.391477 23257 tablet_bootstrap.cc:492] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607: Bootstrap starting.
I20260808 02:13:01.391651 23256 tablet_bootstrap.cc:492] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c: Bootstrap starting.
I20260808 02:13:01.391763 23258 tablet_bootstrap.cc:492] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9: Bootstrap starting.
I20260808 02:13:01.392030 23256 tablet_bootstrap.cc:654] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:01.392184 23258 tablet_bootstrap.cc:654] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:01.392525 23257 tablet_bootstrap.cc:654] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:01.392554 23256 tablet_bootstrap.cc:492] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c: No bootstrap required, opened a new log
I20260808 02:13:01.392598 23256 ts_tablet_manager.cc:1397] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:01.392714 23258 tablet_bootstrap.cc:492] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9: No bootstrap required, opened a new log
I20260808 02:13:01.392707 23256 raft_consensus.cc:348] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.392767 23256 raft_consensus.cc:374] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:01.392765 23258 ts_tablet_manager.cc:1397] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:01.392796 23256 raft_consensus.cc:729] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a449314c494146dc83c02a952a742c9c, State: Initialized, Role: FOLLOWER
I20260808 02:13:01.392838 23256 consensus_queue.cc:260] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.392860 23258 raft_consensus.cc:348] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.392921 23258 raft_consensus.cc:374] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:01.392927 23256 ts_tablet_manager.cc:1428] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:01.392957 23258 raft_consensus.cc:729] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5ee68c7a9844ba99e1c636fe0cd0f9, State: Initialized, Role: FOLLOWER
I20260808 02:13:01.393005 23258 consensus_queue.cc:260] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.393086 23258 ts_tablet_manager.cc:1428] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:01.393126 23257 tablet_bootstrap.cc:492] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607: No bootstrap required, opened a new log
I20260808 02:13:01.393170 23257 ts_tablet_manager.cc:1397] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:01.393287 23257 raft_consensus.cc:348] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.393347 23257 raft_consensus.cc:374] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:01.393365 23257 raft_consensus.cc:729] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456336d0fb6841c8bd4b9e2ef0ecc607, State: Initialized, Role: FOLLOWER
I20260808 02:13:01.393402 23257 consensus_queue.cc:260] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.393491 23257 ts_tablet_manager.cc:1428] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:01.510248 23264 raft_consensus.cc:482] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:01.510330 23264 raft_consensus.cc:504] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.510469 23264 leader_election.cc:288] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a449314c494146dc83c02a952a742c9c (127.19.58.193:45537), 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081)
I20260808 02:13:01.510694 23020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f05df22f2a4181b13b88f9c4bc9b9a" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" is_pre_election: true
I20260808 02:13:01.510720 22578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f05df22f2a4181b13b88f9c4bc9b9a" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a449314c494146dc83c02a952a742c9c" is_pre_election: true
I20260808 02:13:01.510815 23020 raft_consensus.cc:2454] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 0.
I20260808 02:13:01.510815 22578 raft_consensus.cc:2454] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 0.
I20260808 02:13:01.510973 22698 leader_election.cc:302] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a449314c494146dc83c02a952a742c9c, ee5ee68c7a9844ba99e1c636fe0cd0f9; no voters:
I20260808 02:13:01.511091 23264 raft_consensus.cc:2781] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:01.511132 23264 raft_consensus.cc:482] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:01.511145 23264 raft_consensus.cc:3037] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:01.511871 23264 raft_consensus.cc:504] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.511997 23264 leader_election.cc:288] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 election: Requested vote from peers a449314c494146dc83c02a952a742c9c (127.19.58.193:45537), 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081)
I20260808 02:13:01.512146 22578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f05df22f2a4181b13b88f9c4bc9b9a" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a449314c494146dc83c02a952a742c9c"
I20260808 02:13:01.512223 22578 raft_consensus.cc:3037] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:01.512209 23020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f05df22f2a4181b13b88f9c4bc9b9a" candidate_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607"
I20260808 02:13:01.512288 23020 raft_consensus.cc:3037] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:01.512883 23020 raft_consensus.cc:2454] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 1.
I20260808 02:13:01.512961 22578 raft_consensus.cc:2454] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5ee68c7a9844ba99e1c636fe0cd0f9 in term 1.
I20260808 02:13:01.513043 22691 leader_election.cc:302] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456336d0fb6841c8bd4b9e2ef0ecc607, ee5ee68c7a9844ba99e1c636fe0cd0f9; no voters:
I20260808 02:13:01.513159 23264 raft_consensus.cc:2781] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:01.513216 23264 raft_consensus.cc:686] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 LEADER]: Becoming Leader. State: Replica: ee5ee68c7a9844ba99e1c636fe0cd0f9, State: Running, Role: LEADER
I20260808 02:13:01.513265 23264 consensus_queue.cc:237] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } }
I20260808 02:13:01.513911 22335 catalog_manager.cc:5495] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194). New cstate: current_term: 1 leader_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee5ee68c7a9844ba99e1c636fe0cd0f9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:01.539476 22578 raft_consensus.cc:1261] T e0f05df22f2a4181b13b88f9c4bc9b9a P a449314c494146dc83c02a952a742c9c [term 1 FOLLOWER]: Refusing update from remote peer ee5ee68c7a9844ba99e1c636fe0cd0f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:01.539484 23020 raft_consensus.cc:1261] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Refusing update from remote peer ee5ee68c7a9844ba99e1c636fe0cd0f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:01.539695 23264 consensus_queue.cc:1035] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "456336d0fb6841c8bd4b9e2ef0ecc607" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 34081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:01.539839 23275 consensus_queue.cc:1035] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a449314c494146dc83c02a952a742c9c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:13:01.543797 23211 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 02:13:03.059635 22337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:03.059752 22337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:03.060520 22337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4d5e4c05ebd4615a826cb12fe6b3f86: Sending DeleteTablet for 3 replicas of tablet e0f05df22f2a4181b13b88f9c4bc9b9a
I20260808 02:13:03.060755 22554 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f05df22f2a4181b13b88f9c4bc9b9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:03 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:46454
I20260808 02:13:03.060774 22994 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f05df22f2a4181b13b88f9c4bc9b9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:03 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:56250
I20260808 02:13:03.060774 22764 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f05df22f2a4181b13b88f9c4bc9b9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:03 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:47792
I20260808 02:13:03.060899 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 22456
I20260808 02:13:03.060971 23318 tablet_replica.cc:330] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9: stopping tablet replica
I20260808 02:13:03.065224 23318 raft_consensus.cc:2229] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:03.065299 23318 raft_consensus.cc:2258] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:03.065594 23318 ts_tablet_manager.cc:1905] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:03.067005 23315 tablet_replica.cc:330] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607: stopping tablet replica
I20260808 02:13:03.067116 23315 raft_consensus.cc:2229] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:03.067159 23318 ts_tablet_manager.cc:1918] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:13:03.067190 23315 raft_consensus.cc:2258] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:03.067309 23318 log.cc:1196] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/e0f05df22f2a4181b13b88f9c4bc9b9a
I20260808 02:13:03.067751 23318 ts_tablet_manager.cc:1939] T e0f05df22f2a4181b13b88f9c4bc9b9a P ee5ee68c7a9844ba99e1c636fe0cd0f9: Deleting consensus metadata
I20260808 02:13:03.068151 23315 ts_tablet_manager.cc:1905] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:03.068156 22299 catalog_manager.cc:4841] TS ee5ee68c7a9844ba99e1c636fe0cd0f9 (127.19.58.194:38095): tablet e0f05df22f2a4181b13b88f9c4bc9b9a (table test-table [id=27f3c442c7384121b8368771d8428112]) successfully deleted
I20260808 02:13:03.069314 23315 ts_tablet_manager.cc:1918] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 02:13:03.069368 23315 log.cc:1196] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/e0f05df22f2a4181b13b88f9c4bc9b9a
I20260808 02:13:03.069538 23315 ts_tablet_manager.cc:1939] T e0f05df22f2a4181b13b88f9c4bc9b9a P 456336d0fb6841c8bd4b9e2ef0ecc607: Deleting consensus metadata
I20260808 02:13:03.069816 22299 catalog_manager.cc:4841] TS 456336d0fb6841c8bd4b9e2ef0ecc607 (127.19.58.195:34081): tablet e0f05df22f2a4181b13b88f9c4bc9b9a (table test-table [id=27f3c442c7384121b8368771d8428112]) successfully deleted
W20260808 02:13:03.072736 22303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:45537 (error 108)
I20260808 02:13:03.073293 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:03.073115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45537 (local address 127.0.0.1:46460)
0808 02:13:03.073224 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:03.073261 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:45537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":66,"threads_started":1}
I20260808 02:13:03.073324 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 22670
W20260808 02:13:03.073414 22303 catalog_manager.cc:4584] TS a449314c494146dc83c02a952a742c9c (127.19.58.193:45537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e0f05df22f2a4181b13b88f9c4bc9b9a: Network error: Client connection negotiation failed: client connection to 127.19.58.193:45537: connect: Connection refused (error 111)
I20260808 02:13:03.088557 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 22885
I20260808 02:13:03.099121 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 22278
2026-08-08T02:13:03Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5844 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:03 dist-test-slave-1l3l krb5kdc[23323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:03 dist-test-slave-1l3l krb5kdc[23323](info): set up 2 sockets
Aug 08 02:13:03 dist-test-slave-1l3l krb5kdc[23323](info): commencing operation
krb5kdc: starting...
W20260808 02:13:04.221499 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:04 dist-test-slave-1l3l krb5kdc[23323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:13:04.332358 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:38167
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:38167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:04.410555 23346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:04.410751 23346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:04.410772 23346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:04.411908 23346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:04.411949 23346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:04.411962 23346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:04.411973 23346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:04.411984 23346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:04.413403 23346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:38167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:38167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:04.413604 23346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:04.413836 23346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:04.418790 23358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:04.418856 23346 server_base.cc:1034] running on GCE node
W20260808 02:13:04.418804 23356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.418839 23355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:04.419281 23346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:04.419584 23346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:04.420722 23346 hybrid_clock.cc:648] HybridClock initialized: now 1786155184420706 us; error 32 us; skew 500 ppm
Aug 08 02:13:04 dist-test-slave-1l3l krb5kdc[23323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:04.423056 23346 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:13:04.423480 23346 webserver.cc:460] Webserver started at http://127.19.58.254:37671/ using document root <none> and password file <none>
I20260808 02:13:04.423679 23346 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:04.423753 23346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:04.423877 23346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:04.424845 23346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "d59eeec463804fb4a322a964f325ed4b"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "1b12343c33a47bb193256d57f23b8f78"
server_key_iv: "04fa875b67fe52981a40f8ee2eecf3b0"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.425166 23346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "d59eeec463804fb4a322a964f325ed4b"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "1b12343c33a47bb193256d57f23b8f78"
server_key_iv: "04fa875b67fe52981a40f8ee2eecf3b0"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.426450 23346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:04.427058 23371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.427207 23346 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:04.427287 23346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "d59eeec463804fb4a322a964f325ed4b"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "1b12343c33a47bb193256d57f23b8f78"
server_key_iv: "04fa875b67fe52981a40f8ee2eecf3b0"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.427346 23346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:04.461043 23346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:04.463127 23346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:04.463266 23346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:04.467196 23346 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:38167
I20260808 02:13:04.467233 23450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:38167 every 8 connection(s)
I20260808 02:13:04.467566 23346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:04.468426 23451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:04.470629 23451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b: Bootstrap starting.
I20260808 02:13:04.471165 23451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:04.471441 23451 log.cc:824] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:04.472117 23451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b: No bootstrap required, opened a new log
I20260808 02:13:04.473667 23451 raft_consensus.cc:348] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59eeec463804fb4a322a964f325ed4b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38167 } }
I20260808 02:13:04.473799 23451 raft_consensus.cc:374] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:04.473831 23451 raft_consensus.cc:729] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d59eeec463804fb4a322a964f325ed4b, State: Initialized, Role: FOLLOWER
I20260808 02:13:04.473968 23451 consensus_queue.cc:260] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59eeec463804fb4a322a964f325ed4b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38167 } }
I20260808 02:13:04.474028 23451 raft_consensus.cc:388] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:04.474050 23451 raft_consensus.cc:482] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:04.474097 23451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:04.474604 23451 raft_consensus.cc:504] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59eeec463804fb4a322a964f325ed4b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38167 } }
I20260808 02:13:04.474690 23451 leader_election.cc:302] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d59eeec463804fb4a322a964f325ed4b; no voters:
I20260808 02:13:04.474833 23451 leader_election.cc:288] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:04.474898 23456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:04.475057 23451 sys_catalog.cc:564] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:04.475065 23456 raft_consensus.cc:686] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [term 1 LEADER]: Becoming Leader. State: Replica: d59eeec463804fb4a322a964f325ed4b, State: Running, Role: LEADER
I20260808 02:13:04.475204 23456 consensus_queue.cc:237] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59eeec463804fb4a322a964f325ed4b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38167 } }
I20260808 02:13:04.475562 23457 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d59eeec463804fb4a322a964f325ed4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59eeec463804fb4a322a964f325ed4b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38167 } } }
I20260808 02:13:04.475725 23457 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:04.475567 23458 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d59eeec463804fb4a322a964f325ed4b. Latest consensus state: current_term: 1 leader_uuid: "d59eeec463804fb4a322a964f325ed4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59eeec463804fb4a322a964f325ed4b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38167 } } }
I20260808 02:13:04.475965 23458 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:04.476073 23462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:04.476563 23462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:04.476671 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 23346
I20260808 02:13:04.476749 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:04.476907 19691 external_mini_cluster.cc:1428] Setting key 31381e16198e519bb90f477dd811a552
I20260808 02:13:04.478041 23462 catalog_manager.cc:1347] Generated new cluster ID: 1eb1500ce1494552b5cbb19e41966a7f
I20260808 02:13:04.478088 23462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:13:04 dist-test-slave-1l3l krb5kdc[23323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:04.486644 23462 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:04.487232 23462 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:04.490258 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.477285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.19.58.254:38167)
0808 02:13:04.477635 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:04.477645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:04.477742 (+ 97us) server_negotiation.cc:407] Connection header received
0808 02:13:04.477894 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:04.477901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:04.477946 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:04.479831 (+ 1885us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:04.480589 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.481987 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.483077 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.483685 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.483819 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.486480 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:04.486484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:04.486484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:04.486495 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:04.488419 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.488813 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.488816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.488817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.488917 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.489108 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.489109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.489110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.489366 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:04.489513 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:04.489770 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 02:13:04.490017 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":73,"threads_started":1}
I20260808 02:13:04.493214 23462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d59eeec463804fb4a322a964f325ed4b: Generated new TSK 0
I20260808 02:13:04.493353 23462 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:13:04.516405 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38167
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:04.596139 23492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:04.596333 23492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:04.596355 23492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:04.597504 23492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:04.597545 23492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:04.597570 23492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:04.598982 23492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:04.599205 23492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:04.599421 23492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:04.604096 23509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.604125 23506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:04.604187 23492 server_base.cc:1034] running on GCE node
W20260808 02:13:04.604104 23504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:04.604496 23492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:04.604713 23492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:04.605862 23492 hybrid_clock.cc:648] HybridClock initialized: now 1786155184605846 us; error 28 us; skew 500 ppm
Aug 08 02:13:04 dist-test-slave-1l3l krb5kdc[23323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:04.608274 23492 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:13:04.608702 23492 webserver.cc:460] Webserver started at http://127.19.58.193:43211/ using document root <none> and password file <none>
I20260808 02:13:04.608914 23492 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:04.608963 23492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:04.609086 23492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:04.610059 23492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "b094243643df4a2e84dbe3507b25b444"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "ced2b6dcc056c106aeb42da098c58bbc"
server_key_iv: "ca21ef5f22eb813dfad752b9bdedba6c"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.610405 23492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "b094243643df4a2e84dbe3507b25b444"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "ced2b6dcc056c106aeb42da098c58bbc"
server_key_iv: "ca21ef5f22eb813dfad752b9bdedba6c"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.611605 23492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 02:13:04.612303 23517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.612452 23492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.612532 23492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "b094243643df4a2e84dbe3507b25b444"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "ced2b6dcc056c106aeb42da098c58bbc"
server_key_iv: "ca21ef5f22eb813dfad752b9bdedba6c"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.612599 23492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:04.622840 23492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:04.624814 23492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:04.624943 23492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:04.625129 23492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:04.625403 23492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:04.625442 23492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.625469 23492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:04.625494 23492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.631042 23492 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:36373
I20260808 02:13:04.631114 23688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:36373 every 8 connection(s)
I20260808 02:13:04.631383 23492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
Aug 08 02:13:04 dist-test-slave-1l3l krb5kdc[23323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:04.639640 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.631955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:59871 (local address 127.19.58.254:38167)
0808 02:13:04.631997 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:04.631999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:04.632215 (+ 216us) server_negotiation.cc:407] Connection header received
0808 02:13:04.632679 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:04.632681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:04.632693 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:04.632744 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:04.633663 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.634168 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.635109 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.635322 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.635378 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.637345 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:04.637346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:04.637346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:04.637353 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:04.638614 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.638956 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.638957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.638957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.639004 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.639177 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.639178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.639178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.639266 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:04.639307 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:04.639514 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 02:13:04.639577 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:13:04.639775 23692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.631957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:38167 (local address 127.19.58.193:59871)
0808 02:13:04.632110 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:04.632142 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:04.632587 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:04.632808 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:04.632812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:04.633395 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:04.633569 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:04.633573 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.634251 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.634252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:04.635026 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:04.635028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.635393 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.635394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:04.635521 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.635971 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:04.635979 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:04.637233 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:04.638679 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:04.638683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:04.638684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:04.638881 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:04.639049 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:04.639049 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:04.639050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:04.639138 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:04.639306 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:04.639309 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:04.639408 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 02:13:04.639482 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:13:04.639592 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260808 02:13:04.640214 23689 heartbeater.cc:347] Connected to a master server at 127.19.58.254:38167
I20260808 02:13:04.640290 23689 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:04.640425 23689 heartbeater.cc:511] Master 127.19.58.254:38167 requested a full tablet report, sending...
I20260808 02:13:04.640599 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 23492
I20260808 02:13:04.640674 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:04.640838 19691 external_mini_cluster.cc:1428] Setting key e4f89cf6ea7ceb2c849e078ab2efa196
I20260808 02:13:04.640950 23397 ts_manager.cc:194] Registered new tserver with Master: b094243643df4a2e84dbe3507b25b444 (127.19.58.193:36373)
I20260808 02:13:04.641896 23397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:59871
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:13:04.663726 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38167
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:04.741622 23696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:04.741796 23696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:04.741817 23696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:04.742900 23696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:04.742933 23696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:04.742956 23696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:04.744364 23696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:04.744546 23696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:04.744712 23696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:04.749251 23702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.749245 23707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.749248 23705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:04.749514 23696 server_base.cc:1034] running on GCE node
I20260808 02:13:04.749719 23696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:04.749910 23696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:04.751042 23696 hybrid_clock.cc:648] HybridClock initialized: now 1786155184751028 us; error 28 us; skew 500 ppm
Aug 08 02:13:04 dist-test-slave-1l3l krb5kdc[23323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:04.753455 23696 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:13:04.753880 23696 webserver.cc:460] Webserver started at http://127.19.58.194:45045/ using document root <none> and password file <none>
I20260808 02:13:04.754092 23696 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:04.754141 23696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:04.754263 23696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:04.755209 23696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "2f7de69825b44a61b700fdf6846288f8"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "29b50668468c78dd69cff04fb7bee2ea"
server_key_iv: "7558c86cacf119b11a113ec28d6bc8f1"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.755558 23696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "2f7de69825b44a61b700fdf6846288f8"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "29b50668468c78dd69cff04fb7bee2ea"
server_key_iv: "7558c86cacf119b11a113ec28d6bc8f1"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.756821 23696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:04.757524 23717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.757671 23696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:04.757752 23696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "2f7de69825b44a61b700fdf6846288f8"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "29b50668468c78dd69cff04fb7bee2ea"
server_key_iv: "7558c86cacf119b11a113ec28d6bc8f1"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.757819 23696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:04.771265 23696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:04.773255 23696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:04.773391 23696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:04.773578 23696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:04.773838 23696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:04.773872 23696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.773900 23696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:04.773923 23696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.779353 23696 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:41573
I20260808 02:13:04.779428 23888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:41573 every 8 connection(s)
I20260808 02:13:04.779667 23696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
Aug 08 02:13:04 dist-test-slave-1l3l krb5kdc[23323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:04.787433 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.780303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:58459 (local address 127.19.58.254:38167)
0808 02:13:04.780350 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:04.780352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:04.780529 (+ 177us) server_negotiation.cc:407] Connection header received
0808 02:13:04.780946 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:04.780947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:04.780956 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:04.781012 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:04.781868 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.782355 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.783246 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.783452 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.783503 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.785398 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:04.785399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:04.785400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:04.785406 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:04.786448 (+ 1042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.786774 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.786775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.786775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.786817 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.786989 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.786989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.786990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.787076 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:04.787119 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:04.787299 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 02:13:04.787353 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:13:04.787581 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.780274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:38167 (local address 127.19.58.194:58459)
0808 02:13:04.780418 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:04.780455 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:04.780873 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:04.781058 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:04.781060 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:04.781613 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:04.781790 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:04.781794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.782429 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.782430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:04.783169 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:04.783171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.783526 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.783526 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:04.783627 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.784072 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:04.784079 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:04.785314 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:04.786514 (+ 1200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:04.786520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:04.786522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:04.786719 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:04.786860 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:04.786860 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:04.786861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:04.786948 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:04.787115 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:04.787117 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:04.787213 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 02:13:04.787283 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:13:04.787397 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260808 02:13:04.787726 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 23696
I20260808 02:13:04.787823 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:04.788017 19691 external_mini_cluster.cc:1428] Setting key 039f2c426ca652f743e5da659d94c8c0
I20260808 02:13:04.788040 23893 heartbeater.cc:347] Connected to a master server at 127.19.58.254:38167
I20260808 02:13:04.788125 23893 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:04.788290 23893 heartbeater.cc:511] Master 127.19.58.254:38167 requested a full tablet report, sending...
I20260808 02:13:04.788748 23397 ts_manager.cc:194] Registered new tserver with Master: 2f7de69825b44a61b700fdf6846288f8 (127.19.58.194:41573)
I20260808 02:13:04.789160 23397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:58459
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:13:04.810390 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38167
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:04.890058 23899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:04.890242 23899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:04.890264 23899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:04.891372 23899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:04.891413 23899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:04.891440 23899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:04.892885 23899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:04.893098 23899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:04.893293 23899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:04.898098 23912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.898146 23915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:04.898203 23909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:04.898276 23899 server_base.cc:1034] running on GCE node
I20260808 02:13:04.898468 23899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:04.898674 23899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:04.899755 23899 hybrid_clock.cc:648] HybridClock initialized: now 1786155184899736 us; error 32 us; skew 500 ppm
Aug 08 02:13:04 dist-test-slave-1l3l krb5kdc[23323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:04.902094 23899 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:13:04.902513 23899 webserver.cc:460] Webserver started at http://127.19.58.195:45437/ using document root <none> and password file <none>
I20260808 02:13:04.902710 23899 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:04.902753 23899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:04.902880 23899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:04.903893 23899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "7bc1d3ecb6fb4fa99b5161df929c10dc"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "970cd42d80be6bcae4bc9267b30eac5f"
server_key_iv: "585052d5790880a6869e67dae74de91c"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.904197 23899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "7bc1d3ecb6fb4fa99b5161df929c10dc"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "970cd42d80be6bcae4bc9267b30eac5f"
server_key_iv: "585052d5790880a6869e67dae74de91c"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.905386 23899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:04.906011 23928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.906162 23899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:04.906237 23899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "7bc1d3ecb6fb4fa99b5161df929c10dc"
format_stamp: "Formatted at 2026-08-08 02:13:04 on dist-test-slave-1l3l"
server_key: "970cd42d80be6bcae4bc9267b30eac5f"
server_key_iv: "585052d5790880a6869e67dae74de91c"
server_key_version: "encryptionkey@0"
I20260808 02:13:04.906286 23899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:04.938735 23899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:04.940961 23899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:04.941077 23899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:04.941274 23899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:04.941555 23899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:04.941586 23899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.941609 23899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:04.941623 23899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:04.947216 23899 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:42891
I20260808 02:13:04.947297 24117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:42891 every 8 connection(s)
I20260808 02:13:04.947561 23899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
Aug 08 02:13:04 dist-test-slave-1l3l krb5kdc[23323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155184, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:04.955145 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 23899
I20260808 02:13:04.955251 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:04.955458 19691 external_mini_cluster.cc:1428] Setting key bd26fe07aa9441e0ce96b84d99248675
I20260808 02:13:04.956110 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.948272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:57969 (local address 127.19.58.254:38167)
0808 02:13:04.948318 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:04.948321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:04.948530 (+ 209us) server_negotiation.cc:407] Connection header received
0808 02:13:04.948972 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:04.948973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:04.948982 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:04.949030 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:04.949853 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.950357 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.951334 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.951519 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.951562 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.953503 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:04.953504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:04.953504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:04.953511 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:04.954731 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.955087 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.955088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.955089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.955150 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:04.955450 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:04.955451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:04.955452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:04.955581 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:04.955648 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:04.955944 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 02:13:04.956025 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:13:04.956285 24122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:04.948257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:38167 (local address 127.19.58.195:57969)
0808 02:13:04.948420 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:04.948455 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:04.948903 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:04.949083 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:04.949086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:04.949609 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:04.949788 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:04.949793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.950458 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.950459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:04.951259 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:04.951261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:04.951598 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:04.951599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:04.951745 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:04.952121 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:04.952128 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:04.953422 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:04.954804 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:04.954810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:04.954812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:04.955015 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:04.955217 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:04.955218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:04.955219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:04.955353 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:04.955646 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:04.955653 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:04.955818 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 02:13:04.955924 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 02:13:04.956080 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260808 02:13:04.956722 24120 heartbeater.cc:347] Connected to a master server at 127.19.58.254:38167
I20260808 02:13:04.956799 24120 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:04.956965 24120 heartbeater.cc:511] Master 127.19.58.254:38167 requested a full tablet report, sending...
I20260808 02:13:04.957293 23397 ts_manager.cc:194] Registered new tserver with Master: 7bc1d3ecb6fb4fa99b5161df929c10dc (127.19.58.195:42891)
I20260808 02:13:04.957633 23397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:57969
I20260808 02:13:04.967154 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:13:04.978761 23488 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:04.978019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.19.58.254:38167)
0808 02:13:04.978063 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:04.978065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:04.978176 (+ 111us) server_negotiation.cc:407] Connection header received
0808 02:13:04.978216 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:04.978217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:04.978225 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:04.978268 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:04.978702 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40714: BlockingRecv error: recv got EOF from 127.0.0.1:40714 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 02:13:04.979908 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 23492
I20260808 02:13:05.005486 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 23696
I20260808 02:13:05.016985 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 23899
I20260808 02:13:05.023802 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 23346
2026-08-08T02:13:05Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1926 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:05 dist-test-slave-1l3l krb5kdc[24140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:05 dist-test-slave-1l3l krb5kdc[24140](info): set up 2 sockets
Aug 08 02:13:05 dist-test-slave-1l3l krb5kdc[24140](info): commencing operation
krb5kdc: starting...
W20260808 02:13:06.140851 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:06 dist-test-slave-1l3l krb5kdc[24140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:13:06.255820 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42791
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:46381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:42791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:06.334138 24169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:06.334327 24169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:06.334357 24169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:06.335583 24169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:06.335628 24169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:06.335647 24169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:06.335661 24169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:06.335677 24169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:06.337656 24169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:46381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:42791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:06.337904 24169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:06.338173 24169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:06.342607 24178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:06.342635 24176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:06.342799 24169 server_base.cc:1034] running on GCE node
W20260808 02:13:06.342918 24175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:06.343166 24169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:06.343439 24169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:06.344573 24169 hybrid_clock.cc:648] HybridClock initialized: now 1786155186344559 us; error 32 us; skew 500 ppm
W20260808 02:13:07.163130 24169 net_util.cc:504] Time spent look up canonical hostname for localhost 'dist-test-slave-1l3l': real 0.818s user 0.000s sys 0.000s
Aug 08 02:13:07 dist-test-slave-1l3l krb5kdc[24140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155187, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:07.165086 24169 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:13:07.165611 24169 webserver.cc:460] Webserver started at http://127.19.58.254:39243/ using document root <none> and password file <none>
I20260808 02:13:07.165840 24169 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:07.165890 24169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:07.166018 24169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:07.167052 24169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "8578ded85da847819275e20e6487136f"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-1l3l"
server_key: "fff1716bf4a1e2f7ee484b1b6650d39f"
server_key_iv: "ec67bcfb856b4de5ff07201796fbd738"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.167384 24169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "8578ded85da847819275e20e6487136f"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-1l3l"
server_key: "fff1716bf4a1e2f7ee484b1b6650d39f"
server_key_iv: "ec67bcfb856b4de5ff07201796fbd738"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.168716 24169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:13:07.169413 24189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.169580 24169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:07.169672 24169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "8578ded85da847819275e20e6487136f"
format_stamp: "Formatted at 2026-08-08 02:13:07 on dist-test-slave-1l3l"
server_key: "fff1716bf4a1e2f7ee484b1b6650d39f"
server_key_iv: "ec67bcfb856b4de5ff07201796fbd738"
server_key_version: "encryptionkey@0"
I20260808 02:13:07.169726 24169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:07.182165 24169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:07.184605 24169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:07.184706 24169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:07.189095 24169 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:42791
I20260808 02:13:07.189142 24282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:42791 every 8 connection(s)
I20260808 02:13:07.189633 24169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:07.189886 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 24169
I20260808 02:13:07.189988 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:07.190171 19691 external_mini_cluster.cc:1428] Setting key d5db5b41de8bc8ddc46261314c7af9b5
I20260808 02:13:07.190335 24283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:07.193389 24283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f: Bootstrap starting.
I20260808 02:13:07.194106 24283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:07.194403 24283 log.cc:824] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:07.195281 24283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f: No bootstrap required, opened a new log
Aug 08 02:13:07 dist-test-slave-1l3l krb5kdc[24140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:07.197587 24283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8578ded85da847819275e20e6487136f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42791 } }
I20260808 02:13:07.197718 24283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:07.197751 24283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8578ded85da847819275e20e6487136f, State: Initialized, Role: FOLLOWER
I20260808 02:13:07.197871 24283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8578ded85da847819275e20e6487136f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42791 } }
I20260808 02:13:07.197942 24283 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:07.197973 24283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:07.198029 24283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:07.198513 24283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8578ded85da847819275e20e6487136f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42791 } }
I20260808 02:13:07.198619 24283 leader_election.cc:302] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8578ded85da847819275e20e6487136f; no voters:
I20260808 02:13:07.198771 24283 leader_election.cc:288] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:07.198817 24291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:07.198983 24291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [term 1 LEADER]: Becoming Leader. State: Replica: 8578ded85da847819275e20e6487136f, State: Running, Role: LEADER
I20260808 02:13:07.199045 24283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:07.199071 24291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8578ded85da847819275e20e6487136f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42791 } }
I20260808 02:13:07.199445 24291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8578ded85da847819275e20e6487136f. Latest consensus state: current_term: 1 leader_uuid: "8578ded85da847819275e20e6487136f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8578ded85da847819275e20e6487136f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42791 } } }
I20260808 02:13:07.199738 24291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:07.199635 24293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8578ded85da847819275e20e6487136f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8578ded85da847819275e20e6487136f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42791 } } }
I20260808 02:13:07.199995 24293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:07.200019 24308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:07.200672 24308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:07.200971 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.190635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.19.58.254:42791)
0808 02:13:07.190822 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.190832 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.190856 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:13:07.191386 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.191392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.191436 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.191545 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:07.192347 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.193233 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.194221 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.194698 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.194791 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.197312 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:07.197316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:07.197316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:07.197325 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:07.199048 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:07.199452 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:07.199455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:07.199455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:07.199553 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:07.199792 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:07.199793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:07.199793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:07.200082 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:07.200220 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.200644 (+ 424us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.200773 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260808 02:13:07.201828 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:13:07.202116 24308 catalog_manager.cc:1347] Generated new cluster ID: 61a6dc33cb24497e944c979129bc43e9
I20260808 02:13:07.202153 24308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:07.210412 24308 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:07.210923 24308 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:07.215238 24308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8578ded85da847819275e20e6487136f: Generated new TSK 0
I20260808 02:13:07.215409 24308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:07.218204 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:07.214540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51936 (local address 127.19.58.254:42791)
0808 02:13:07.214573 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:07.214575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:07.214701 (+ 126us) server_negotiation.cc:407] Connection header received
0808 02:13:07.214752 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:07.214752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:07.214761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:07.214824 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:07.215230 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.216058 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.216771 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:07.217081 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:07.217147 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:07.217772 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:07.217773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:07.217773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:07.217977 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:07.217994 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:07.218001 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:07.218062 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:07.218106 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 02:13:07.218130 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:13:07.220547 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 24169
2026-08-08T02:13:07Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2194 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:07 dist-test-slave-1l3l krb5kdc[24331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:07 dist-test-slave-1l3l krb5kdc[24331](info): set up 2 sockets
Aug 08 02:13:07 dist-test-slave-1l3l krb5kdc[24331](info): commencing operation
krb5kdc: starting...
W20260808 02:13:08.328366 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:08 dist-test-slave-1l3l krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:13:08.440438 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:38267
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:33545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:38267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:08.519045 24355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:08.519241 24355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:08.519271 24355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:08.520401 24355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:08.520443 24355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:08.520462 24355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:08.520478 24355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:08.520493 24355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:08.521947 24355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:33545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:38267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:38267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:08.522171 24355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:08.522364 24355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:08.526883 24365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:08.527006 24355 server_base.cc:1034] running on GCE node
W20260808 02:13:08.526863 24367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.526862 24363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:08.527364 24355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:08.527630 24355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:08.528777 24355 hybrid_clock.cc:648] HybridClock initialized: now 1786155188528774 us; error 39 us; skew 500 ppm
Aug 08 02:13:08 dist-test-slave-1l3l krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155188, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:08.531172 24355 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:13:08.531616 24355 webserver.cc:460] Webserver started at http://127.19.58.254:36283/ using document root <none> and password file <none>
I20260808 02:13:08.531844 24355 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:08.531901 24355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:08.532021 24355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:08.533025 24355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "94df93e825624030ae66b7c082420bbf"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-1l3l"
server_key: "905133f5cafb5d98efe2da7bd1861f68"
server_key_iv: "5a2aef6d795f03b979995b118427fdc2"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.533371 24355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "94df93e825624030ae66b7c082420bbf"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-1l3l"
server_key: "905133f5cafb5d98efe2da7bd1861f68"
server_key_iv: "5a2aef6d795f03b979995b118427fdc2"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.534612 24355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:08.535305 24376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.535486 24355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:08.535568 24355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "94df93e825624030ae66b7c082420bbf"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-1l3l"
server_key: "905133f5cafb5d98efe2da7bd1861f68"
server_key_iv: "5a2aef6d795f03b979995b118427fdc2"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.535633 24355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:08.556089 24355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:08.558036 24355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:08.558166 24355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:08.562036 24355 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:38267
I20260808 02:13:08.562074 24446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:38267 every 8 connection(s)
I20260808 02:13:08.562376 24355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:08.563282 24447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:08.564983 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 24355
I20260808 02:13:08.565074 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:08.565233 19691 external_mini_cluster.cc:1428] Setting key ba7b19dfe0d177b2c5c8f051fbac3542
I20260808 02:13:08.565667 24447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf: Bootstrap starting.
I20260808 02:13:08.566231 24447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:08.566512 24447 log.cc:824] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:08.567145 24447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf: No bootstrap required, opened a new log
I20260808 02:13:08.568902 24447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94df93e825624030ae66b7c082420bbf" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38267 } }
I20260808 02:13:08.569028 24447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:08.569060 24447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94df93e825624030ae66b7c082420bbf, State: Initialized, Role: FOLLOWER
I20260808 02:13:08.569173 24447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94df93e825624030ae66b7c082420bbf" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38267 } }
I20260808 02:13:08.569244 24447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:08.569276 24447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:08.569371 24447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:08.570029 24447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94df93e825624030ae66b7c082420bbf" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38267 } }
I20260808 02:13:08.570143 24447 leader_election.cc:302] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94df93e825624030ae66b7c082420bbf; no voters:
I20260808 02:13:08.570334 24447 leader_election.cc:288] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:08.570358 24453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:08.570509 24453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [term 1 LEADER]: Becoming Leader. State: Replica: 94df93e825624030ae66b7c082420bbf, State: Running, Role: LEADER
I20260808 02:13:08.570576 24447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:08.570680 24453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94df93e825624030ae66b7c082420bbf" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38267 } }
Aug 08 02:13:08 dist-test-slave-1l3l krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:08.572073 24455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94df93e825624030ae66b7c082420bbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94df93e825624030ae66b7c082420bbf" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38267 } } }
W20260808 02:13:08.572374 24478 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 02:13:08.572683 24455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [sys.catalog]: This master's current role is: LEADER
W20260808 02:13:08.572700 24478 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:08.572402 24454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94df93e825624030ae66b7c082420bbf. Latest consensus state: current_term: 1 leader_uuid: "94df93e825624030ae66b7c082420bbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94df93e825624030ae66b7c082420bbf" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38267 } } }
I20260808 02:13:08.572762 24454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:08.572934 24482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:08.573084 24482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:08.574615 24482 catalog_manager.cc:1347] Generated new cluster ID: 531459aaf8f045528606f113d6a7480a
I20260808 02:13:08.574673 24482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:08.575583 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.565701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40096 (local address 127.19.58.254:38267)
0808 02:13:08.565869 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.565878 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.565921 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:13:08.566348 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.566356 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.566407 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.566529 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.567111 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:08.568073 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:08.568791 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:08.569255 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:08.569358 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:08.572029 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.572033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.572034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:08.572045 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.573894 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:08.574253 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:08.574256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:08.574256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:08.574363 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:08.574610 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:08.574611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:08.574611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:08.574942 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.575089 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.575227 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.575376 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260808 02:13:08.586385 24482 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:08.586843 24482 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:08.592904 24482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94df93e825624030ae66b7c082420bbf: Generated new TSK 0
I20260808 02:13:08.593070 24482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:13:08.620154 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38267
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:33545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:08.698427 24487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:08.698617 24487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:08.698638 24487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:08.699753 24487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:08.699790 24487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:08.699817 24487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:08.701251 24487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:33545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:08.701478 24487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:08.701692 24487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:08.706341 24503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.706367 24496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.706363 24498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:08.706379 24487 server_base.cc:1034] running on GCE node
I20260808 02:13:08.706694 24487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:08.706877 24487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:08.708009 24487 hybrid_clock.cc:648] HybridClock initialized: now 1786155188707997 us; error 30 us; skew 500 ppm
Aug 08 02:13:08 dist-test-slave-1l3l krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155188, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:08.710306 24487 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:13:08.710697 24487 webserver.cc:460] Webserver started at http://127.19.58.193:45431/ using document root <none> and password file <none>
I20260808 02:13:08.710880 24487 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:08.710927 24487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:08.711016 24487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:08.711979 24487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "159b8960cd55416693889bba6e52240b"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-1l3l"
server_key: "aee86a2ebb7c2d8036b49ab34506448e"
server_key_iv: "1ea7e5a4753096cf7c6fcdd7a1b14b5e"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.712260 24487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "159b8960cd55416693889bba6e52240b"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-1l3l"
server_key: "aee86a2ebb7c2d8036b49ab34506448e"
server_key_iv: "1ea7e5a4753096cf7c6fcdd7a1b14b5e"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.713425 24487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:08.714030 24518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.714222 24487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:08.714308 24487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "159b8960cd55416693889bba6e52240b"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-1l3l"
server_key: "aee86a2ebb7c2d8036b49ab34506448e"
server_key_iv: "1ea7e5a4753096cf7c6fcdd7a1b14b5e"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.714368 24487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:08.750941 24487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:08.752664 24487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:08.752805 24487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:08.753021 24487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:08.753309 24487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:08.753343 24487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.753371 24487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:08.753388 24487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.758808 24487 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:34747
I20260808 02:13:08.758869 24684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:34747 every 8 connection(s)
I20260808 02:13:08.759155 24487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
Aug 08 02:13:08 dist-test-slave-1l3l krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155188, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:08.764864 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 24487
I20260808 02:13:08.764986 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:08.765172 19691 external_mini_cluster.cc:1428] Setting key 84c24004915607aa1c9eb0996f2c6ea4
I20260808 02:13:08.767995 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.759786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:56961 (local address 127.19.58.254:38267)
0808 02:13:08.759831 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.759833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.760034 (+ 201us) server_negotiation.cc:407] Connection header received
0808 02:13:08.760544 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.760545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.760558 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.760610 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.761510 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:08.762013 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:08.762981 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:08.763182 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:08.763237 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:08.765293 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.765294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.765295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:08.765302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.766740 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:08.767098 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:08.767099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:08.767099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:08.767161 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:08.767350 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:08.767351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:08.767351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:08.767488 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.767552 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.767862 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.767932 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:13:08.768245 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.759772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:38267 (local address 127.19.58.193:56961)
0808 02:13:08.759921 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.759955 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:08.760444 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:08.760677 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:08.760681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:08.761238 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:08.761417 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:08.761421 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:08.762094 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:08.762095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:08.762890 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:08.762899 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:08.763253 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:08.763254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:08.763371 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:08.763835 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:08.763843 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:08.765210 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:08.766813 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:08.766820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:08.766821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:08.767035 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:08.767217 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:08.767218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:08.767218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:08.767303 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:08.767551 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:08.767558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:08.767739 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 02:13:08.767850 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 02:13:08.768034 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260808 02:13:08.768714 24685 heartbeater.cc:347] Connected to a master server at 127.19.58.254:38267
I20260808 02:13:08.768810 24685 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:08.768985 24685 heartbeater.cc:511] Master 127.19.58.254:38267 requested a full tablet report, sending...
I20260808 02:13:08.769497 24401 ts_manager.cc:194] Registered new tserver with Master: 159b8960cd55416693889bba6e52240b (127.19.58.193:34747)
I20260808 02:13:08.770354 24401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:56961
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:13:08.788302 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38267
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:33545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:08.866472 24694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:08.866659 24694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:08.866681 24694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:08.867766 24694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:08.867799 24694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:08.867823 24694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:08.869184 24694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:33545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:08.869355 24694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:08.869536 24694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:08.874410 24709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.874424 24705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:08.874476 24703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:08.874568 24694 server_base.cc:1034] running on GCE node
I20260808 02:13:08.874768 24694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:08.874996 24694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:08.876128 24694 hybrid_clock.cc:648] HybridClock initialized: now 1786155188876110 us; error 32 us; skew 500 ppm
Aug 08 02:13:08 dist-test-slave-1l3l krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155188, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:08.878525 24694 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:13:08.878896 24694 webserver.cc:460] Webserver started at http://127.19.58.194:36399/ using document root <none> and password file <none>
I20260808 02:13:08.879058 24694 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:08.879096 24694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:08.879186 24694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:08.880151 24694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "29e0a0cf4b59418c9f2637054673096d"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-1l3l"
server_key: "2d8668351ab0bde8cfc76939855ac377"
server_key_iv: "83b5425dadb5fe97b6a625c6318e88c2"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.880446 24694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "29e0a0cf4b59418c9f2637054673096d"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-1l3l"
server_key: "2d8668351ab0bde8cfc76939855ac377"
server_key_iv: "83b5425dadb5fe97b6a625c6318e88c2"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.881593 24694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:08.882205 24724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.882362 24694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:08.882453 24694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "29e0a0cf4b59418c9f2637054673096d"
format_stamp: "Formatted at 2026-08-08 02:13:08 on dist-test-slave-1l3l"
server_key: "2d8668351ab0bde8cfc76939855ac377"
server_key_iv: "83b5425dadb5fe97b6a625c6318e88c2"
server_key_version: "encryptionkey@0"
I20260808 02:13:08.882504 24694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:08.898582 24694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:08.900264 24694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:08.900368 24694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:08.900565 24694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:08.900801 24694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:08.900831 24694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.900851 24694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:08.900867 24694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:08.905951 24694 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:44407
I20260808 02:13:08.905988 24888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:44407 every 8 connection(s)
I20260808 02:13:08.906260 24694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
Aug 08 02:13:08 dist-test-slave-1l3l krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155188, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:08.912554 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 24694
I20260808 02:13:08.912662 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:08.912840 19691 external_mini_cluster.cc:1428] Setting key 07ac421f309a97c2e5ed4313af70e95d
I20260808 02:13:08.914439 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.906870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:37569 (local address 127.19.58.254:38267)
0808 02:13:08.906926 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:08.906928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:08.907106 (+ 178us) server_negotiation.cc:407] Connection header received
0808 02:13:08.907542 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:08.907542 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:08.907551 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:08.907602 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:08.908461 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:08.908950 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:08.909825 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:08.910005 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:08.910047 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:08.911947 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:08.911948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:08.911948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:08.911953 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:08.913230 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:08.913585 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:08.913586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:08.913586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:08.913645 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:08.913850 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:08.913851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:08.913852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:08.913965 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:08.914020 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:08.914260 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 02:13:08.914340 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:13:08.914582 24891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:08.906824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:38267 (local address 127.19.58.194:37569)
0808 02:13:08.906993 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:08.907031 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:08.907470 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:08.907649 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:08.907652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:08.908210 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:08.908394 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:08.908399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:08.909010 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:08.909010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:08.909768 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:08.909771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:08.910063 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:08.910064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:08.910159 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:08.910574 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:08.910582 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:08.911870 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:08.913292 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:08.913299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:08.913301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:08.913518 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:08.913699 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:08.913700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:08.913700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:08.913794 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:08.914019 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:08.914026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:08.914145 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 02:13:08.914231 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:13:08.914371 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260808 02:13:08.915437 24889 heartbeater.cc:347] Connected to a master server at 127.19.58.254:38267
I20260808 02:13:08.915521 24889 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:08.915676 24889 heartbeater.cc:511] Master 127.19.58.254:38267 requested a full tablet report, sending...
I20260808 02:13:08.916041 24401 ts_manager.cc:194] Registered new tserver with Master: 29e0a0cf4b59418c9f2637054673096d (127.19.58.194:44407)
I20260808 02:13:08.916411 24401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:37569
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:13:08.936061 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38267
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:33545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:09.015453 24897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:09.015648 24897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:09.015669 24897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:09.016799 24897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:09.016842 24897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:09.016871 24897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:09.018294 24897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:33545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:09.018507 24897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:09.018714 24897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:09.023365 24907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:09.023444 24897 server_base.cc:1034] running on GCE node
W20260808 02:13:09.023330 24909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:09.023418 24906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:09.023739 24897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:09.023984 24897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:09.025099 24897 hybrid_clock.cc:648] HybridClock initialized: now 1786155189025092 us; error 28 us; skew 500 ppm
Aug 08 02:13:09 dist-test-slave-1l3l krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155189, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:09.027459 24897 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:13:09.027894 24897 webserver.cc:460] Webserver started at http://127.19.58.195:44339/ using document root <none> and password file <none>
I20260808 02:13:09.028126 24897 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:09.028187 24897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:09.028319 24897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:09.029547 24897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "4fc9bd8976b049deb55b01e0acedf1e9"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-1l3l"
server_key: "38386f97ae9ebcf5480eb459345c39b4"
server_key_iv: "a644e2d62c4808fccf3525b826e92f91"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.029961 24897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "4fc9bd8976b049deb55b01e0acedf1e9"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-1l3l"
server_key: "38386f97ae9ebcf5480eb459345c39b4"
server_key_iv: "a644e2d62c4808fccf3525b826e92f91"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.034678 24897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260808 02:13:09.035645 24922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:09.035843 24897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:09.035988 24897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "4fc9bd8976b049deb55b01e0acedf1e9"
format_stamp: "Formatted at 2026-08-08 02:13:09 on dist-test-slave-1l3l"
server_key: "38386f97ae9ebcf5480eb459345c39b4"
server_key_iv: "a644e2d62c4808fccf3525b826e92f91"
server_key_version: "encryptionkey@0"
I20260808 02:13:09.036123 24897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:09.052479 24897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:09.054071 24897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:09.054194 24897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:09.054390 24897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:09.054677 24897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:09.054713 24897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:09.054734 24897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:09.054757 24897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:09.060456 24897 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:37133
I20260808 02:13:09.060513 25083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:37133 every 8 connection(s)
I20260808 02:13:09.060791 24897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:09.064817 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 24897
I20260808 02:13:09.064891 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:09.065059 19691 external_mini_cluster.cc:1428] Setting key 121245bd84b496df62249e731e76139e
Aug 08 02:13:09 dist-test-slave-1l3l krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155189, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:09.069443 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:09.061461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:57071 (local address 127.19.58.254:38267)
0808 02:13:09.061515 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:09.061517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:09.061684 (+ 167us) server_negotiation.cc:407] Connection header received
0808 02:13:09.062179 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:09.062180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:09.062192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:09.062246 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:09.063159 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.063683 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.064671 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.064865 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.064937 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:09.067260 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:09.067261 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:09.067261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:09.067268 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:09.068447 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:09.068778 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:09.068779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:09.068779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:09.068822 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:09.068998 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:09.068998 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:09.068999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:09.069088 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:09.069130 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:09.069326 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 02:13:09.069379 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:13:09.069604 25087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:09.061421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:38267 (local address 127.19.58.195:57071)
0808 02:13:09.061565 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:09.061605 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:09.062078 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:09.062315 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:09.062319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:09.062874 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:09.063060 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:09.063064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.063794 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.063795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:09.064586 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:09.064588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.065086 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.065088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:09.065232 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:09.065748 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:09.065757 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:09.067182 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:09.068509 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:09.068515 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:09.068517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:09.068723 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:09.068865 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:09.068865 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:09.068866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:09.068957 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:09.069129 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:09.069132 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:09.069235 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 02:13:09.069309 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:13:09.069424 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260808 02:13:09.070022 25085 heartbeater.cc:347] Connected to a master server at 127.19.58.254:38267
I20260808 02:13:09.070112 25085 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:09.070299 25085 heartbeater.cc:511] Master 127.19.58.254:38267 requested a full tablet report, sending...
I20260808 02:13:09.070640 24401 ts_manager.cc:194] Registered new tserver with Master: 4fc9bd8976b049deb55b01e0acedf1e9 (127.19.58.195:37133)
I20260808 02:13:09.070987 24401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:57071
I20260808 02:13:09.076617 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:13:09 dist-test-slave-1l3l krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155189, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 02:13:09 dist-test-slave-1l3l krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155189, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:09.098245 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:09.091936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40098 (local address 127.19.58.254:38267)
0808 02:13:09.091986 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:09.091988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:09.092141 (+ 153us) server_negotiation.cc:407] Connection header received
0808 02:13:09.092192 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:09.092193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:09.092204 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:09.092261 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:09.092802 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.093414 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.094123 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:09.094296 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:09.094337 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:09.096396 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:09.096398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:09.096398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:09.096404 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:09.097441 (+ 1037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:09.097707 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:09.097708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:09.097708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:09.097749 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:09.097853 (+ 104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:09.097854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:09.097854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:09.097986 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:09.098057 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:09.098124 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 02:13:09.098180 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 02:13:09.098455 24401 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40098
I20260808 02:13:09.098901 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 24487
I20260808 02:13:09.110946 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 24694
I20260808 02:13:09.123176 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 24897
I20260808 02:13:09.133937 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 24355
2026-08-08T02:13:09Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1913 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:09 dist-test-slave-1l3l krb5kdc[25103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:09 dist-test-slave-1l3l krb5kdc[25103](info): set up 2 sockets
Aug 08 02:13:09 dist-test-slave-1l3l krb5kdc[25103](info): commencing operation
krb5kdc: starting...
W20260808 02:13:10.244778 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:13:10.360618 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:39483
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:39483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:10.440724 25121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:10.440920 25121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:10.440951 25121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:10.442058 25121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:10.442098 25121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:10.442117 25121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:10.442133 25121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:10.442147 25121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:10.443634 25121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:39483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:39483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:10.443890 25121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:10.444094 25121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:10.448624 25129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:10.448755 25121 server_base.cc:1034] running on GCE node
W20260808 02:13:10.448644 25132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:10.448630 25134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:10.449106 25121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:10.449342 25121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:10.450489 25121 hybrid_clock.cc:648] HybridClock initialized: now 1786155190450471 us; error 29 us; skew 500 ppm
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:10.452901 25121 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:13:10.453305 25121 webserver.cc:460] Webserver started at http://127.19.58.254:34841/ using document root <none> and password file <none>
I20260808 02:13:10.453536 25121 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:10.453588 25121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:10.453711 25121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:10.454713 25121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "7ce061fef6224b5da5d6621bdede6be4"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "e94a8fdece8e82d5483ff41bec912bde"
server_key_iv: "800b82d3ad0feec595407806c35a2d15"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.455042 25121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "7ce061fef6224b5da5d6621bdede6be4"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "e94a8fdece8e82d5483ff41bec912bde"
server_key_iv: "800b82d3ad0feec595407806c35a2d15"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.456331 25121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:10.457018 25146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.457161 25121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:10.457242 25121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "7ce061fef6224b5da5d6621bdede6be4"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "e94a8fdece8e82d5483ff41bec912bde"
server_key_iv: "800b82d3ad0feec595407806c35a2d15"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.457299 25121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:10.475955 25121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:10.477725 25121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:10.477864 25121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:10.481890 25121 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:39483
I20260808 02:13:10.481940 25223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:39483 every 8 connection(s)
I20260808 02:13:10.482525 25121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:10.483220 25224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:10.485097 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 25121
I20260808 02:13:10.485186 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:10.485358 19691 external_mini_cluster.cc:1428] Setting key c360a5f4e4a4a8ff6215de31c6bb01f4
I20260808 02:13:10.485684 25224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4: Bootstrap starting.
I20260808 02:13:10.486410 25224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:10.486737 25224 log.cc:824] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:10.487527 25224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4: No bootstrap required, opened a new log
I20260808 02:13:10.489857 25224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce061fef6224b5da5d6621bdede6be4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39483 } }
I20260808 02:13:10.490023 25224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:10.490057 25224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce061fef6224b5da5d6621bdede6be4, State: Initialized, Role: FOLLOWER
I20260808 02:13:10.490180 25224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce061fef6224b5da5d6621bdede6be4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39483 } }
I20260808 02:13:10.490253 25224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:10.490283 25224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:10.490345 25224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:10.490880 25224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce061fef6224b5da5d6621bdede6be4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39483 } }
I20260808 02:13:10.490979 25224 leader_election.cc:302] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ce061fef6224b5da5d6621bdede6be4; no voters:
I20260808 02:13:10.491143 25224 leader_election.cc:288] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:10.491184 25229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:10.491292 25229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [term 1 LEADER]: Becoming Leader. State: Replica: 7ce061fef6224b5da5d6621bdede6be4, State: Running, Role: LEADER
I20260808 02:13:10.491425 25224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:10.491385 25229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce061fef6224b5da5d6621bdede6be4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39483 } }
I20260808 02:13:10.491796 25229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ce061fef6224b5da5d6621bdede6be4. Latest consensus state: current_term: 1 leader_uuid: "7ce061fef6224b5da5d6621bdede6be4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce061fef6224b5da5d6621bdede6be4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39483 } } }
I20260808 02:13:10.491881 25229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:10.491845 25234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ce061fef6224b5da5d6621bdede6be4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce061fef6224b5da5d6621bdede6be4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39483 } } }
I20260808 02:13:10.491937 25234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:10.492087 25239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:10.492606 25239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:10.494354 25239 catalog_manager.cc:1347] Generated new cluster ID: f82c834404ae4d01995295f7a18d4a1b
I20260808 02:13:10.494418 25239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:10.496887 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.485840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.19.58.254:39483)
0808 02:13:10.486031 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:10.486041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:10.486078 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:13:10.486634 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:10.486643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:10.486707 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:10.486862 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:10.487543 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.488935 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.489674 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.490260 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.490367 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.493168 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:10.493172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:10.493172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:10.493184 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:10.495127 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.495600 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.495604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.495604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.495740 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.495994 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.495995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.495995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.496282 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:10.496437 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:10.496545 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 02:13:10.496675 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260808 02:13:10.504159 25239 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:10.504660 25239 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:10.519289 25239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ce061fef6224b5da5d6621bdede6be4: Generated new TSK 0
I20260808 02:13:10.519510 25239 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:13:10.546159 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39483
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:10.626262 25267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:10.626466 25267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:10.626487 25267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:10.627563 25267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:10.627599 25267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:10.627624 25267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:10.629078 25267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:10.629253 25267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:10.629429 25267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:10.634146 25273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:10.634176 25275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:10.634146 25279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:10.634435 25267 server_base.cc:1034] running on GCE node
I20260808 02:13:10.634621 25267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:10.634816 25267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:10.635946 25267 hybrid_clock.cc:648] HybridClock initialized: now 1786155190635937 us; error 32 us; skew 500 ppm
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:10.638375 25267 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:13:10.638753 25267 webserver.cc:460] Webserver started at http://127.19.58.193:33969/ using document root <none> and password file <none>
I20260808 02:13:10.638927 25267 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:10.638967 25267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:10.639058 25267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:10.640065 25267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "36e8d1bfda574b67aa5c3ba41c00c77e"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "821db6f1854e07c2e74ba27e1f08217d"
server_key_iv: "dd6cd0ba3b1a6e71f688e6bd3a2d8c1b"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.640362 25267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "36e8d1bfda574b67aa5c3ba41c00c77e"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "821db6f1854e07c2e74ba27e1f08217d"
server_key_iv: "dd6cd0ba3b1a6e71f688e6bd3a2d8c1b"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.641564 25267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:10.642203 25291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.642329 25267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:10.642407 25267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "36e8d1bfda574b67aa5c3ba41c00c77e"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "821db6f1854e07c2e74ba27e1f08217d"
server_key_iv: "dd6cd0ba3b1a6e71f688e6bd3a2d8c1b"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.642472 25267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:10.652119 25267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:10.653954 25267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:10.654088 25267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:10.654289 25267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:10.654573 25267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:10.654606 25267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.654627 25267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:10.654652 25267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.660245 25267 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:34457
I20260808 02:13:10.660303 25455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:34457 every 8 connection(s)
I20260808 02:13:10.660588 25267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:10.660900 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 25267
I20260808 02:13:10.661002 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:10.661206 19691 external_mini_cluster.cc:1428] Setting key a8379cdbaf642de8cd61885435220b57
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:10.670248 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.661332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:54955 (local address 127.19.58.254:39483)
0808 02:13:10.661394 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:10.661396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:10.661560 (+ 164us) server_negotiation.cc:407] Connection header received
0808 02:13:10.662626 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:10.662627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:10.662637 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:10.662684 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:10.663436 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.664025 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.665048 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.665313 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.665379 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.667600 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:10.667601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:10.667601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:10.667608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:10.668852 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.669301 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.669302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.669303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.669390 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.669657 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.669657 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.669657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.669754 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:10.669799 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:10.670117 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 02:13:10.670182 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
I20260808 02:13:10.670477 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.661287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39483 (local address 127.19.58.193:54955)
0808 02:13:10.661452 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:10.661493 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:10.662520 (+ 1027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:10.662761 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:10.662765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:10.663081 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:10.663340 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:10.663346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.664117 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.664118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:10.664961 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:10.664964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.665415 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.665416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:10.665525 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.665992 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:10.666000 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:10.667509 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:10.668934 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:10.668942 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:10.668944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:10.669225 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:10.669454 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:10.669455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:10.669456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:10.669593 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:10.669819 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:10.669826 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:10.669988 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 02:13:10.670106 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 02:13:10.670262 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260808 02:13:10.671031 25458 heartbeater.cc:347] Connected to a master server at 127.19.58.254:39483
I20260808 02:13:10.671115 25458 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:10.671307 25458 heartbeater.cc:511] Master 127.19.58.254:39483 requested a full tablet report, sending...
I20260808 02:13:10.671805 25169 ts_manager.cc:194] Registered new tserver with Master: 36e8d1bfda574b67aa5c3ba41c00c77e (127.19.58.193:34457)
I20260808 02:13:10.672513 25169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:54955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:13:10.686174 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39483
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:10.765215 25466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:10.765412 25466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:10.765443 25466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:10.766547 25466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:10.766587 25466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:10.766620 25466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:10.768057 25466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:10.768278 25466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:10.768487 25466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:10.773080 25471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:10.773089 25475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:10.773089 25472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:10.773435 25466 server_base.cc:1034] running on GCE node
I20260808 02:13:10.773605 25466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:10.773784 25466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:10.774937 25466 hybrid_clock.cc:648] HybridClock initialized: now 1786155190774924 us; error 26 us; skew 500 ppm
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:10.777360 25466 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:13:10.777750 25466 webserver.cc:460] Webserver started at http://127.19.58.194:40173/ using document root <none> and password file <none>
I20260808 02:13:10.777940 25466 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:10.777995 25466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:10.778118 25466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:10.779098 25466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "889608ccf9874591b9879feb811b43a9"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "0d1e762ffc2123bee13b57178f285430"
server_key_iv: "f49a8c5864fea21e606ef9cf76b097d3"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.779415 25466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "889608ccf9874591b9879feb811b43a9"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "0d1e762ffc2123bee13b57178f285430"
server_key_iv: "f49a8c5864fea21e606ef9cf76b097d3"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.780680 25466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:13:10.781432 25487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.781582 25466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:10.781664 25466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "889608ccf9874591b9879feb811b43a9"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "0d1e762ffc2123bee13b57178f285430"
server_key_iv: "f49a8c5864fea21e606ef9cf76b097d3"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.781720 25466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:10.797921 25466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:10.799615 25466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:10.799803 25466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:10.800019 25466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:10.800294 25466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:10.800328 25466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.800357 25466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:10.800379 25466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.805682 25466 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:36877
I20260808 02:13:10.805740 25658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:36877 every 8 connection(s)
I20260808 02:13:10.806020 25466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:10.810793 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 25466
I20260808 02:13:10.810930 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:10.811111 19691 external_mini_cluster.cc:1428] Setting key 27345c05d60b0994cb117d3da5027e1a
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:10.815145 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.806661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:59509 (local address 127.19.58.254:39483)
0808 02:13:10.806713 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:10.806715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:10.806899 (+ 184us) server_negotiation.cc:407] Connection header received
0808 02:13:10.807816 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:10.807818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:10.807831 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:10.807883 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:10.808468 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.808976 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.809909 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.810112 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.810169 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.812470 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:10.812471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:10.812472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:10.812480 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:10.813807 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.814268 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.814268 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.814269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.814320 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.814566 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.814566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.814567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.814664 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:10.814709 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:10.814975 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 02:13:10.815056 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:13:10.815196 25661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.806642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39483 (local address 127.19.58.194:59509)
0808 02:13:10.806779 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:10.806813 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:10.807690 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:10.807968 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:10.807971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:10.808207 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:10.808378 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:10.808382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.809057 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.809058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:10.809823 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:10.809825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.810180 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.810181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:10.810292 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.810766 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:10.810776 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:10.812364 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:10.813907 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:10.813914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:10.813915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:10.814189 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:10.814373 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:10.814373 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:10.814374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:10.814504 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:10.814711 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:10.814715 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:10.814833 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 02:13:10.814922 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 02:13:10.815037 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260808 02:13:10.815618 25659 heartbeater.cc:347] Connected to a master server at 127.19.58.254:39483
I20260808 02:13:10.815734 25659 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:10.815936 25659 heartbeater.cc:511] Master 127.19.58.254:39483 requested a full tablet report, sending...
I20260808 02:13:10.816320 25169 ts_manager.cc:194] Registered new tserver with Master: 889608ccf9874591b9879feb811b43a9 (127.19.58.194:36877)
I20260808 02:13:10.816682 25169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:59509
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:13:10.835558 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39483
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:10.914232 25666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:10.914431 25666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:10.914463 25666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:10.915586 25666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:10.915627 25666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:10.915659 25666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:10.917104 25666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:10.917327 25666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:10.917536 25666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:10.922158 25674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:10.922168 25675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:10.922370 25666 server_base.cc:1034] running on GCE node
W20260808 02:13:10.922159 25682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:10.922639 25666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:10.922847 25666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:10.923974 25666 hybrid_clock.cc:648] HybridClock initialized: now 1786155190923960 us; error 29 us; skew 500 ppm
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:10.926450 25666 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:13:10.926851 25666 webserver.cc:460] Webserver started at http://127.19.58.195:43645/ using document root <none> and password file <none>
I20260808 02:13:10.927043 25666 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:10.927091 25666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:10.927211 25666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:10.928270 25666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "0910bd89e4114795830df4a6162f55ab"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "cd5a197dfe5fdf7e96a97e7449a05855"
server_key_iv: "0e01fa3d9b0f8d7bddb93e579f272f41"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.928594 25666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "0910bd89e4114795830df4a6162f55ab"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "cd5a197dfe5fdf7e96a97e7449a05855"
server_key_iv: "0e01fa3d9b0f8d7bddb93e579f272f41"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.929832 25666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:10.930498 25693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.930701 25666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:10.930787 25666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "0910bd89e4114795830df4a6162f55ab"
format_stamp: "Formatted at 2026-08-08 02:13:10 on dist-test-slave-1l3l"
server_key: "cd5a197dfe5fdf7e96a97e7449a05855"
server_key_iv: "0e01fa3d9b0f8d7bddb93e579f272f41"
server_key_version: "encryptionkey@0"
I20260808 02:13:10.930845 25666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:10.941962 25666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:10.943974 25666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:10.944101 25666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:10.944290 25666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:10.944595 25666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:10.944638 25666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.944666 25666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:10.944689 25666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:10.950382 25666 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:45185
I20260808 02:13:10.950413 25866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:45185 every 8 connection(s)
I20260808 02:13:10.950686 25666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:10.958855 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.951320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:51105 (local address 127.19.58.254:39483)
0808 02:13:10.951368 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:10.951370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:10.951569 (+ 199us) server_negotiation.cc:407] Connection header received
0808 02:13:10.952533 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:10.952534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:10.952544 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:10.952597 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:10.953124 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.953612 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.954519 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.954704 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.954753 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.956562 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:10.956563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:10.956563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:10.956570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:10.957819 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.958168 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.958169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.958169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.958215 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.958411 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.958412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.958412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.958504 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:10.958552 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:10.958742 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 02:13:10.958793 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:13:10.959017 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.951280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39483 (local address 127.19.58.195:51105)
0808 02:13:10.951461 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:10.951499 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:10.952443 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:10.952655 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:10.952658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:10.952882 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:10.953062 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:10.953065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.953682 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.953682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:10.954455 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:10.954456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.954760 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.954760 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:10.954847 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.955203 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:10.955210 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:10.956487 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:10.957888 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:10.957894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:10.957895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:10.958111 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:10.958257 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:10.958258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:10.958258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:10.958349 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:10.958549 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:10.958551 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:10.958649 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 02:13:10.958725 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:13:10.958828 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260808 02:13:10.959460 25867 heartbeater.cc:347] Connected to a master server at 127.19.58.254:39483
I20260808 02:13:10.959538 25867 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:10.959694 25867 heartbeater.cc:511] Master 127.19.58.254:39483 requested a full tablet report, sending...
I20260808 02:13:10.959924 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 25666
I20260808 02:13:10.960040 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:10.960219 19691 external_mini_cluster.cc:1428] Setting key e7703357d475f554bc83545e638a727f
I20260808 02:13:10.960225 25169 ts_manager.cc:194] Registered new tserver with Master: 0910bd89e4114795830df4a6162f55ab (127.19.58.195:45185)
I20260808 02:13:10.960693 25169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:51105
I20260808 02:13:10.961071 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:13:10.983337 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.976327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57822 (local address 127.19.58.193:34457)
0808 02:13:10.976480 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:10.976484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:10.976492 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:10.976515 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:10.976516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:10.976530 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:10.976591 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:10.977150 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.977854 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.978570 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.978901 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.978959 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.980985 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:10.980995 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:10.980995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:10.981002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:10.982270 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.982605 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.982608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.982608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.982660 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.982820 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.982821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.982821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.982949 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:10.983074 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:10.983154 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 02:13:10.983231 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
Aug 08 02:13:10 dist-test-slave-1l3l krb5kdc[25103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:10.990435 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:10.984596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59860 (local address 127.19.58.254:39483)
0808 02:13:10.984658 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:10.984660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:10.984669 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:10.984728 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:10.984729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:10.984738 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:10.984779 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:10.985200 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.985689 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.986305 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:10.986505 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:10.986557 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:10.988459 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:10.988461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:10.988461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:10.988468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:10.989473 (+ 1005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.989747 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.989748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.989749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.989799 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:10.989956 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:10.989957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:10.989958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:10.990096 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:10.990178 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:10.990279 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 02:13:10.990350 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 02:13:10.990643 25168 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59860
I20260808 02:13:10.990926 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 25267
I20260808 02:13:11.007948 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 25466
I20260808 02:13:11.020639 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 25666
I20260808 02:13:11.037505 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 25121
2026-08-08T02:13:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1905 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:11 dist-test-slave-1l3l krb5kdc[25886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:11 dist-test-slave-1l3l krb5kdc[25886](info): set up 2 sockets
Aug 08 02:13:11 dist-test-slave-1l3l krb5kdc[25886](info): commencing operation
krb5kdc: starting...
W20260808 02:13:12.156029 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:12 dist-test-slave-1l3l krb5kdc[25886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:13:12.265990 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40787
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:40787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:12.343817 25911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:12.344018 25911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:12.344050 25911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:12.345244 25911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:12.345288 25911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:12.345306 25911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:12.345321 25911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:12.345337 25911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:12.346782 25911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:40787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:12.347015 25911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:12.347254 25911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:12.351848 25917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:12.351853 25922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:12.351936 25920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:12.352097 25911 server_base.cc:1034] running on GCE node
I20260808 02:13:12.352315 25911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:12.352595 25911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:12.353730 25911 hybrid_clock.cc:648] HybridClock initialized: now 1786155192353701 us; error 61 us; skew 500 ppm
W20260808 02:13:12.973107 25911 net_util.cc:504] Time spent look up canonical hostname for localhost 'dist-test-slave-1l3l': real 0.619s user 0.000s sys 0.000s
Aug 08 02:13:12 dist-test-slave-1l3l krb5kdc[25886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155192, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:12.974982 25911 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:13:12.975375 25911 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:12.975435 25911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:12.975567 25911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:12.976709 25911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "94d3c9a87d75410f860fa3cd5966f4c4"
format_stamp: "Formatted at 2026-08-08 02:13:12 on dist-test-slave-1l3l"
server_key: "ecc006aa3f25cad855a785f16ffc0e0b"
server_key_iv: "a02b1fd909950a25e6c88dbc534ec71c"
server_key_version: "encryptionkey@0"
I20260808 02:13:12.977092 25911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "94d3c9a87d75410f860fa3cd5966f4c4"
format_stamp: "Formatted at 2026-08-08 02:13:12 on dist-test-slave-1l3l"
server_key: "ecc006aa3f25cad855a785f16ffc0e0b"
server_key_iv: "a02b1fd909950a25e6c88dbc534ec71c"
server_key_version: "encryptionkey@0"
I20260808 02:13:12.978364 25911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:12.979087 25928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:12.979243 25911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:12.979331 25911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "94d3c9a87d75410f860fa3cd5966f4c4"
format_stamp: "Formatted at 2026-08-08 02:13:12 on dist-test-slave-1l3l"
server_key: "ecc006aa3f25cad855a785f16ffc0e0b"
server_key_iv: "a02b1fd909950a25e6c88dbc534ec71c"
server_key_version: "encryptionkey@0"
I20260808 02:13:12.979451 25911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:12.992635 25911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:12.994697 25911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:12.994812 25911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:12.999073 25911 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:40787
I20260808 02:13:12.999096 25999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:40787 every 8 connection(s)
I20260808 02:13:12.999361 25911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:13.000350 26003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:13.002557 26003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4: Bootstrap starting.
I20260808 02:13:13.003129 26003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:13.003405 26003 log.cc:824] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:13.004062 26003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4: No bootstrap required, opened a new log
I20260808 02:13:13.005621 26003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d3c9a87d75410f860fa3cd5966f4c4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40787 } }
I20260808 02:13:13.005751 26003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:13.005774 26003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d3c9a87d75410f860fa3cd5966f4c4, State: Initialized, Role: FOLLOWER
I20260808 02:13:13.005915 26003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d3c9a87d75410f860fa3cd5966f4c4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40787 } }
I20260808 02:13:13.005981 26003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:13.006001 26003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:13.006032 26003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:13.006788 26003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d3c9a87d75410f860fa3cd5966f4c4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40787 } }
I20260808 02:13:13.006875 26003 leader_election.cc:302] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94d3c9a87d75410f860fa3cd5966f4c4; no voters:
I20260808 02:13:13.007014 26003 leader_election.cc:288] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:13.007066 26007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:13.007232 26007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [term 1 LEADER]: Becoming Leader. State: Replica: 94d3c9a87d75410f860fa3cd5966f4c4, State: Running, Role: LEADER
I20260808 02:13:13.007454 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 25911
I20260808 02:13:13.007548 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:13.007740 19691 external_mini_cluster.cc:1428] Setting key c6ea2c80150fe0f27f8dafdb45d62421
I20260808 02:13:13.007259 26003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:13.007335 26007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d3c9a87d75410f860fa3cd5966f4c4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40787 } }
I20260808 02:13:13.008538 26011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94d3c9a87d75410f860fa3cd5966f4c4. Latest consensus state: current_term: 1 leader_uuid: "94d3c9a87d75410f860fa3cd5966f4c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d3c9a87d75410f860fa3cd5966f4c4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40787 } } }
I20260808 02:13:13.008630 26011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:13.008826 26008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94d3c9a87d75410f860fa3cd5966f4c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d3c9a87d75410f860fa3cd5966f4c4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40787 } } }
I20260808 02:13:13.008919 26008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:13.009173 26024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:13.009689 26024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:13.011227 26024 catalog_manager.cc:1347] Generated new cluster ID: c90f95dd5c824f1491f2139d6846b92b
I20260808 02:13:13.011276 26024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:13:13 dist-test-slave-1l3l krb5kdc[25886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:13.016779 26024 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:13.017346 26024 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:13.019531 26018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.008499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56156 (local address 127.19.58.254:40787)
0808 02:13:13.008692 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.008701 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.008737 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:13:13.008865 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.008872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.008945 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.010098 (+ 1153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.010189 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.011419 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.012478 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.013106 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.013202 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.015935 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.015939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.015940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.015952 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.018031 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.018398 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.018401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.018402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.018511 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.018685 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.018686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.018686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.018961 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.019117 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.019182 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.019323 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260808 02:13:13.031771 26024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4: Generated new TSK 0
I20260808 02:13:13.031986 26024 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:13:13.064078 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40787
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:13.157007 26039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:13.157199 26039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:13.157222 26039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:13.158353 26039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:13.158399 26039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:13.158427 26039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:13.159997 26039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:13.160214 26039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:13.160426 26039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:13.165151 26051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.165163 26049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:13.165228 26039 server_base.cc:1034] running on GCE node
W20260808 02:13:13.165207 26047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:13.165520 26039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:13.165724 26039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:13.166860 26039 hybrid_clock.cc:648] HybridClock initialized: now 1786155193166845 us; error 33 us; skew 500 ppm
Aug 08 02:13:13 dist-test-slave-1l3l krb5kdc[25886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.169199 26039 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:13:13.169535 26039 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:13.169587 26039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:13.169709 26039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:13.170678 26039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "f68a003b78b54c65b1b5c4452cf12b85"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "b35ac7eb391873bd844b757b92a34397"
server_key_iv: "6217b1b0100e3278609d5ffe4a22d1c1"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.170993 26039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "f68a003b78b54c65b1b5c4452cf12b85"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "b35ac7eb391873bd844b757b92a34397"
server_key_iv: "6217b1b0100e3278609d5ffe4a22d1c1"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.172207 26039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 02:13:13.172834 26063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.172982 26039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.173060 26039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "f68a003b78b54c65b1b5c4452cf12b85"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "b35ac7eb391873bd844b757b92a34397"
server_key_iv: "6217b1b0100e3278609d5ffe4a22d1c1"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.173116 26039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:13.196278 26039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:13.197961 26039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:13.198067 26039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:13.198231 26039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:13.198463 26039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:13.198493 26039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.198511 26039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:13.198529 26039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.204165 26039 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:45101
I20260808 02:13:13.204198 26233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:45101 every 8 connection(s)
I20260808 02:13:13.204454 26039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:13.209803 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 26039
I20260808 02:13:13.209923 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:13.210088 19691 external_mini_cluster.cc:1428] Setting key 9970edc113325997ae615f51b88969bd
Aug 08 02:13:13 dist-test-slave-1l3l krb5kdc[25886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:13.213249 26018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.205106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:34367 (local address 127.19.58.254:40787)
0808 02:13:13.205179 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.205182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.205337 (+ 155us) server_negotiation.cc:407] Connection header received
0808 02:13:13.205781 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.205782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.205793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.205839 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.206704 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.207209 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.208208 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.208524 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.208582 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.210670 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.210671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.210672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.210679 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.211937 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.212358 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.212358 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.212359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.212461 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.212675 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.212676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.212677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.212804 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.212861 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.213071 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.213165 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:13:13.213299 26236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.205071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40787 (local address 127.19.58.193:34367)
0808 02:13:13.205228 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.205263 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.205704 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.205913 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.205916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.206150 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:13.206626 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.206631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.207290 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.207291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.208112 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.208114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.208615 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.208616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.208720 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.209150 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:13.209158 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:13.210576 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:13.212011 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.212019 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.212021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.212290 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.212525 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.212526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.212526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.212623 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.212859 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:13.212861 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:13.212965 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.213038 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.213149 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260808 02:13:13.213992 26234 heartbeater.cc:347] Connected to a master server at 127.19.58.254:40787
I20260808 02:13:13.214087 26234 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:13.214227 26234 heartbeater.cc:511] Master 127.19.58.254:40787 requested a full tablet report, sending...
I20260808 02:13:13.214709 25948 ts_manager.cc:194] Registered new tserver with Master: f68a003b78b54c65b1b5c4452cf12b85 (127.19.58.193:45101)
I20260808 02:13:13.215318 25948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:34367
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:13:13.232633 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40787
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:13.311353 26240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:13.311553 26240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:13.311586 26240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:13.312721 26240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:13.312762 26240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:13.312798 26240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:13.314263 26240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:13.314493 26240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:13.314700 26240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:13.319417 26247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.319403 26252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.319519 26248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:13.319730 26240 server_base.cc:1034] running on GCE node
I20260808 02:13:13.319937 26240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:13.320153 26240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:13.321307 26240 hybrid_clock.cc:648] HybridClock initialized: now 1786155193321291 us; error 28 us; skew 500 ppm
Aug 08 02:13:13 dist-test-slave-1l3l krb5kdc[25886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.323750 26240 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:13:13.324059 26240 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:13.324110 26240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:13.324234 26240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:13.325222 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "4cc7c218c8484c2eb25c6fd40cd54623"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "0ebdfa0e84671b7b12a7ec801ca6c4f3"
server_key_iv: "70f30204ca488d3d5da60faac4d9556b"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.325543 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "4cc7c218c8484c2eb25c6fd40cd54623"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "0ebdfa0e84671b7b12a7ec801ca6c4f3"
server_key_iv: "70f30204ca488d3d5da60faac4d9556b"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.326762 26240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:13.327348 26259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.327503 26240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:13.327584 26240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "4cc7c218c8484c2eb25c6fd40cd54623"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "0ebdfa0e84671b7b12a7ec801ca6c4f3"
server_key_iv: "70f30204ca488d3d5da60faac4d9556b"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.327639 26240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:13.348541 26240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:13.350675 26240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:13.350807 26240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:13.351006 26240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:13.351301 26240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:13.351336 26240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.351363 26240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:13.351397 26240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.356860 26240 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:41721
I20260808 02:13:13.356896 26427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:41721 every 8 connection(s)
I20260808 02:13:13.357142 26240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
Aug 08 02:13:13 dist-test-slave-1l3l krb5kdc[25886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:13.366143 26018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.357786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:42263 (local address 127.19.58.254:40787)
0808 02:13:13.357841 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.357843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.358013 (+ 170us) server_negotiation.cc:407] Connection header received
0808 02:13:13.358449 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.358450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.358460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.358509 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.359332 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.359833 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.360815 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.361018 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.361073 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.363083 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.363084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.363085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.363109 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.364759 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.365197 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.365199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.365199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.365293 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.365542 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.365543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.365544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.365662 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.365719 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.365968 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.366051 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 02:13:13.366285 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.357747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40787 (local address 127.19.58.194:42263)
0808 02:13:13.357903 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.357936 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.358366 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.358561 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.358564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.358795 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:13.359263 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.359268 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.359903 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.359904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.360726 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.360728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.361080 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.361081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.361186 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.361573 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:13.361581 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:13.362982 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:13.364832 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.364839 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.364841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.365129 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.365350 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.365351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.365352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.365489 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.365717 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:13.365724 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:13.365850 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.365937 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.366083 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260808 02:13:13.366729 26428 heartbeater.cc:347] Connected to a master server at 127.19.58.254:40787
I20260808 02:13:13.366822 26428 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:13.366978 26428 heartbeater.cc:511] Master 127.19.58.254:40787 requested a full tablet report, sending...
I20260808 02:13:13.367051 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 26240
I20260808 02:13:13.367148 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:13.367350 19691 external_mini_cluster.cc:1428] Setting key 2497d024ae4d3151388dc6aa368ceed9
I20260808 02:13:13.367364 25948 ts_manager.cc:194] Registered new tserver with Master: 4cc7c218c8484c2eb25c6fd40cd54623 (127.19.58.194:41721)
I20260808 02:13:13.367740 25948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:42263
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:13:13.389889 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40787
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:13.468490 26434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:13.468675 26434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:13.468698 26434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:13.469801 26434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:13.469839 26434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:13.469867 26434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:13.471309 26434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:13.471516 26434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:13.471748 26434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:13.476336 26443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.476435 26442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.476354 26447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:13.476435 26434 server_base.cc:1034] running on GCE node
I20260808 02:13:13.476737 26434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:13.476969 26434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:13.478139 26434 hybrid_clock.cc:648] HybridClock initialized: now 1786155193478124 us; error 37 us; skew 500 ppm
Aug 08 02:13:13 dist-test-slave-1l3l krb5kdc[25886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:13.480448 26434 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:13:13.480719 26434 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:13.480760 26434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:13.480849 26434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:13.481873 26434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "69d669b25b784875852e7036dc72a30f"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "c8e7d3e010b9a9f2264bf5a4e600313b"
server_key_iv: "047f6c905d392c3f6e598bf4449e7c30"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.482170 26434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "69d669b25b784875852e7036dc72a30f"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "c8e7d3e010b9a9f2264bf5a4e600313b"
server_key_iv: "047f6c905d392c3f6e598bf4449e7c30"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.483395 26434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:13.484072 26456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.484217 26434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.484297 26434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "69d669b25b784875852e7036dc72a30f"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "c8e7d3e010b9a9f2264bf5a4e600313b"
server_key_iv: "047f6c905d392c3f6e598bf4449e7c30"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.484356 26434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:13.508736 26434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:13.510641 26434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:13.510774 26434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:13.510995 26434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:13.511286 26434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:13.511320 26434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.511349 26434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:13.511373 26434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.517047 26434 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:45431
I20260808 02:13:13.517108 26620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:45431 every 8 connection(s)
I20260808 02:13:13.517323 26434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
Aug 08 02:13:13 dist-test-slave-1l3l krb5kdc[25886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155193, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:13.524230 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 26434
I20260808 02:13:13.524304 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:13.524456 19691 external_mini_cluster.cc:1428] Setting key e2cdf9ca3a9383d80c61df8ecc2a1b11
I20260808 02:13:13.525684 26018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.517980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:55069 (local address 127.19.58.254:40787)
0808 02:13:13.518031 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.518034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.518202 (+ 168us) server_negotiation.cc:407] Connection header received
0808 02:13:13.518690 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.518691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.518701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.518745 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.519609 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.520163 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.521125 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.521311 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.521359 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.523192 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.523193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.523193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.523200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.524497 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.524842 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.524843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.524844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.524899 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.525107 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.525108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.525108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.525218 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.525274 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.525521 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.525598 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:13:13.525835 26625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.517955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40787 (local address 127.19.58.195:55069)
0808 02:13:13.518096 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.518129 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.518614 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.518801 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.518804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.519044 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:13.519523 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.519528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.520242 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.520243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.521055 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.521057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.521395 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.521395 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.521487 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.521861 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:13.521868 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:13.523115 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:13.524563 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.524569 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.524571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.524776 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.524953 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:13.524953 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:13.524954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:13.525053 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:13.525273 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:13.525280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:13.525406 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.525490 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.525631 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260808 02:13:13.526252 26621 heartbeater.cc:347] Connected to a master server at 127.19.58.254:40787
I20260808 02:13:13.526338 26621 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:13.526480 26621 heartbeater.cc:511] Master 127.19.58.254:40787 requested a full tablet report, sending...
I20260808 02:13:13.526790 25947 ts_manager.cc:194] Registered new tserver with Master: 69d669b25b784875852e7036dc72a30f (127.19.58.195:45431)
I20260808 02:13:13.527145 25947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:55069
I20260808 02:13:13.535907 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:13:13.542282 26018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.537060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.19.58.254:40787)
0808 02:13:13.537118 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.537119 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.537211 (+ 92us) server_negotiation.cc:407] Connection header received
0808 02:13:13.537250 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.537251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.537259 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.537356 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.537816 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.538278 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.539029 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.539210 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.539274 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.540291 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.540292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.540293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:13.540298 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.541477 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.541722 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.541723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.541723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.541766 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:13.541883 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:13.541883 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:13.541883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:13.542006 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.542073 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.542148 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.542204 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:13:13.543622 25947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:13.545159 25947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:13.551005 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.547156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45101 (local address 127.0.0.1:48340)
0808 02:13:13.547305 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.547346 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.547420 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.547642 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.547645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.547649 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:13.547731 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.547734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.548762 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.548763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.549863 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.549865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.550711 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.550712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.550804 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.550818 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.550876 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.550938 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260808 02:13:13.551016 26643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.547283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48340 (local address 127.19.58.193:45101)
0808 02:13:13.547469 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.547473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.547484 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:13:13.547530 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.547532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.547547 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.547626 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:13.547824 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.548677 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.549949 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.550643 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.550702 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.550757 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.550897 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.550937 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
I20260808 02:13:13.551560 26645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.547359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:45431 (local address 127.0.0.1:34526)
0808 02:13:13.547746 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.547756 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.547806 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.548085 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.548086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.548088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:13.548128 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.548130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.549531 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.549532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.550421 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.550424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.551329 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.551330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.551390 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.551393 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.551440 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.551479 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":99,"threads_started":1}
I20260808 02:13:13.551560 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.547168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:41721 (local address 127.0.0.1:57258)
0808 02:13:13.547582 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.547591 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.547668 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.547903 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.547903 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.547905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:13.547944 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.547946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.549234 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.549235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.550346 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.550349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.551354 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.551354 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.551416 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.551419 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.551466 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.551494 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":88,"threads_started":1}
I20260808 02:13:13.551611 26644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.547345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.19.58.194:41721)
0808 02:13:13.547540 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.547545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.547642 (+ 97us) server_negotiation.cc:407] Connection header received
0808 02:13:13.547805 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.547807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.547823 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.547912 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:13.547999 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.549050 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.550419 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.551226 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.551280 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.551327 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.551489 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.551533 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":88,"threads_started":1}
I20260808 02:13:13.551672 26137 tablet_service.cc:1467] Processing CreateTablet for tablet 488f26973aa449d0ad9c2d6520d46d9e (DEFAULT_TABLE table=test-table [id=bdc90c1431c24adc9490edeb78c4a952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:13.551867 26646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.547570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34526 (local address 127.19.58.195:45431)
0808 02:13:13.547759 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.547763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.547799 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:13:13.547863 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.547865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.547878 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.547946 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:13.548202 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.549354 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.550511 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.551175 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.551241 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.551293 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.551746 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.551790 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":85,"threads_started":1}
I20260808 02:13:13.552055 26137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 488f26973aa449d0ad9c2d6520d46d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:13.552371 26325 tablet_service.cc:1467] Processing CreateTablet for tablet 488f26973aa449d0ad9c2d6520d46d9e (DEFAULT_TABLE table=test-table [id=bdc90c1431c24adc9490edeb78c4a952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:13.552736 26325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 488f26973aa449d0ad9c2d6520d46d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:13.554541 26647 tablet_bootstrap.cc:492] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85: Bootstrap starting.
I20260808 02:13:13.555089 26648 tablet_bootstrap.cc:492] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: Bootstrap starting.
I20260808 02:13:13.555052 26528 tablet_service.cc:1467] Processing CreateTablet for tablet 488f26973aa449d0ad9c2d6520d46d9e (DEFAULT_TABLE table=test-table [id=bdc90c1431c24adc9490edeb78c4a952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:13.555189 26647 tablet_bootstrap.cc:654] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:13.555409 26528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 488f26973aa449d0ad9c2d6520d46d9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:13.555831 26648 tablet_bootstrap.cc:654] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:13.555812 26647 log.cc:824] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:13.556363 26648 log.cc:824] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:13.556655 26647 tablet_bootstrap.cc:492] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85: No bootstrap required, opened a new log
I20260808 02:13:13.556732 26647 ts_tablet_manager.cc:1397] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:13:13.556999 26648 tablet_bootstrap.cc:492] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: No bootstrap required, opened a new log
I20260808 02:13:13.557058 26648 ts_tablet_manager.cc:1397] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 02:13:13.558473 26656 tablet_bootstrap.cc:492] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: Bootstrap starting.
I20260808 02:13:13.558647 26647 raft_consensus.cc:348] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 } } peers { permanent_uuid: "69d669b25b784875852e7036dc72a30f" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45431 } } peers { permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 } }
I20260808 02:13:13.558645 26648 raft_consensus.cc:348] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 } } peers { permanent_uuid: "69d669b25b784875852e7036dc72a30f" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45431 } } peers { permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 } }
I20260808 02:13:13.558773 26647 raft_consensus.cc:374] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:13.558789 26648 raft_consensus.cc:374] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:13.558794 26647 raft_consensus.cc:729] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f68a003b78b54c65b1b5c4452cf12b85, State: Initialized, Role: FOLLOWER
I20260808 02:13:13.558835 26648 raft_consensus.cc:729] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc7c218c8484c2eb25c6fd40cd54623, State: Initialized, Role: FOLLOWER
I20260808 02:13:13.558934 26647 consensus_queue.cc:260] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 } } peers { permanent_uuid: "69d669b25b784875852e7036dc72a30f" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45431 } } peers { permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 } }
I20260808 02:13:13.558961 26648 consensus_queue.cc:260] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 } } peers { permanent_uuid: "69d669b25b784875852e7036dc72a30f" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45431 } } peers { permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 } }
I20260808 02:13:13.559223 26648 ts_tablet_manager.cc:1428] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:13:13.559242 26234 heartbeater.cc:503] Master 127.19.58.254:40787 was elected leader, sending a full tablet report...
I20260808 02:13:13.559319 26656 tablet_bootstrap.cc:654] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:13.559358 26428 heartbeater.cc:503] Master 127.19.58.254:40787 was elected leader, sending a full tablet report...
I20260808 02:13:13.559229 26647 ts_tablet_manager.cc:1428] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 02:13:13.559943 26656 log.cc:824] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:13.560611 26656 tablet_bootstrap.cc:492] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: No bootstrap required, opened a new log
I20260808 02:13:13.560665 26656 ts_tablet_manager.cc:1397] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:13.562039 26656 raft_consensus.cc:348] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 } } peers { permanent_uuid: "69d669b25b784875852e7036dc72a30f" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45431 } } peers { permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 } }
I20260808 02:13:13.562150 26656 raft_consensus.cc:374] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:13.562168 26656 raft_consensus.cc:729] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69d669b25b784875852e7036dc72a30f, State: Initialized, Role: FOLLOWER
I20260808 02:13:13.562239 26656 consensus_queue.cc:260] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 } } peers { permanent_uuid: "69d669b25b784875852e7036dc72a30f" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45431 } } peers { permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 } }
I20260808 02:13:13.562430 26656 ts_tablet_manager.cc:1428] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:13.562482 26621 heartbeater.cc:503] Master 127.19.58.254:40787 was elected leader, sending a full tablet report...
W20260808 02:13:13.607642 26429 tablet.cc:2369] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:13:13.705166 26235 tablet.cc:2369] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:13.754175 26662 raft_consensus.cc:482] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:13.754305 26662 raft_consensus.cc:504] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 } } peers { permanent_uuid: "69d669b25b784875852e7036dc72a30f" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45431 } } peers { permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 } }
I20260808 02:13:13.754674 26662 leader_election.cc:288] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f68a003b78b54c65b1b5c4452cf12b85 (127.19.58.193:45101), 4cc7c218c8484c2eb25c6fd40cd54623 (127.19.58.194:41721)
I20260808 02:13:13.758603 26644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.754989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:49367 (local address 127.19.58.194:41721)
0808 02:13:13.755061 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.755063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.755073 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:13:13.755100 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.755101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.755108 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.755170 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:13.755327 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.756108 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.757650 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.758204 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.758277 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.758285 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.758495 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.758531 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:13:13.758605 26643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.754987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:47943 (local address 127.19.58.193:45101)
0808 02:13:13.755042 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.755045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.755169 (+ 124us) server_negotiation.cc:407] Connection header received
0808 02:13:13.755196 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.755197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.755207 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.755246 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:13.755442 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.756147 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.757645 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.758204 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.758277 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.758285 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.758495 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.758531 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:13:13.758628 26625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.754860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:41721 (local address 127.19.58.195:49367)
0808 02:13:13.754915 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.754931 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.754999 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.755182 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.755182 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.755184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:13.755254 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.755255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.756209 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.756210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.757557 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.757561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.758282 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.758283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.758425 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.758427 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.758485 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.758538 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":145}
I20260808 02:13:13.758634 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.754883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45101 (local address 127.19.58.195:47943)
0808 02:13:13.755083 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.755092 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:13.755140 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:13.755259 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:13.755259 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:13.755261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:13.755371 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.755373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.756213 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.756213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.757557 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:13.757561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.758283 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.758283 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:13.758419 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.758424 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:13:13.758485 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 02:13:13.758538 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":16,"thread_start_us":123,"threads_started":1}
I20260808 02:13:13.759053 26165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "488f26973aa449d0ad9c2d6520d46d9e" candidate_uuid: "69d669b25b784875852e7036dc72a30f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68a003b78b54c65b1b5c4452cf12b85" is_pre_election: true
I20260808 02:13:13.759039 26354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "488f26973aa449d0ad9c2d6520d46d9e" candidate_uuid: "69d669b25b784875852e7036dc72a30f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" is_pre_election: true
I20260808 02:13:13.759207 26165 raft_consensus.cc:2454] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69d669b25b784875852e7036dc72a30f in term 0.
I20260808 02:13:13.759207 26354 raft_consensus.cc:2454] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69d669b25b784875852e7036dc72a30f in term 0.
I20260808 02:13:13.759420 26461 leader_election.cc:302] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cc7c218c8484c2eb25c6fd40cd54623, 69d669b25b784875852e7036dc72a30f; no voters:
I20260808 02:13:13.759538 26662 raft_consensus.cc:2781] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:13.759593 26662 raft_consensus.cc:482] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:13.759621 26662 raft_consensus.cc:3037] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:13.760360 26662 raft_consensus.cc:504] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 } } peers { permanent_uuid: "69d669b25b784875852e7036dc72a30f" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45431 } } peers { permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 } }
I20260808 02:13:13.760489 26662 leader_election.cc:288] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [CANDIDATE]: Term 1 election: Requested vote from peers f68a003b78b54c65b1b5c4452cf12b85 (127.19.58.193:45101), 4cc7c218c8484c2eb25c6fd40cd54623 (127.19.58.194:41721)
I20260808 02:13:13.760629 26165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "488f26973aa449d0ad9c2d6520d46d9e" candidate_uuid: "69d669b25b784875852e7036dc72a30f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f68a003b78b54c65b1b5c4452cf12b85"
I20260808 02:13:13.760653 26354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "488f26973aa449d0ad9c2d6520d46d9e" candidate_uuid: "69d669b25b784875852e7036dc72a30f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cc7c218c8484c2eb25c6fd40cd54623"
I20260808 02:13:13.760701 26165 raft_consensus.cc:3037] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:13.760738 26354 raft_consensus.cc:3037] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:13.761381 26165 raft_consensus.cc:2454] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69d669b25b784875852e7036dc72a30f in term 1.
I20260808 02:13:13.761540 26354 raft_consensus.cc:2454] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69d669b25b784875852e7036dc72a30f in term 1.
I20260808 02:13:13.761550 26457 leader_election.cc:302] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69d669b25b784875852e7036dc72a30f, f68a003b78b54c65b1b5c4452cf12b85; no voters:
I20260808 02:13:13.761808 26662 raft_consensus.cc:2781] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:13.761952 26662 raft_consensus.cc:686] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 1 LEADER]: Becoming Leader. State: Replica: 69d669b25b784875852e7036dc72a30f, State: Running, Role: LEADER
I20260808 02:13:13.762032 26662 consensus_queue.cc:237] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 } } peers { permanent_uuid: "69d669b25b784875852e7036dc72a30f" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45431 } } peers { permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 } }
I20260808 02:13:13.762889 25947 catalog_manager.cc:5495] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f reported cstate change: term changed from 0 to 1, leader changed from <none> to 69d669b25b784875852e7036dc72a30f (127.19.58.195). New cstate: current_term: 1 leader_uuid: "69d669b25b784875852e7036dc72a30f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69d669b25b784875852e7036dc72a30f" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 } health_report { overall_health: UNKNOWN } } }
W20260808 02:13:13.767794 26624 tablet.cc:2369] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:13.777155 26646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.775082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34532 (local address 127.19.58.195:45431)
0808 02:13:13.775130 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.775131 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.775140 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:13.775167 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.775167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.775182 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.775237 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:13.775345 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.775852 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.776596 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:13.776774 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:13.776821 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:13.776925 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:13.777011 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:13.777064 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.777075 (+ 11us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.777091 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 02:13:13.778615 26165 raft_consensus.cc:1261] T 488f26973aa449d0ad9c2d6520d46d9e P f68a003b78b54c65b1b5c4452cf12b85 [term 1 FOLLOWER]: Refusing update from remote peer 69d669b25b784875852e7036dc72a30f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:13.778615 26354 raft_consensus.cc:1261] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623 [term 1 FOLLOWER]: Refusing update from remote peer 69d669b25b784875852e7036dc72a30f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:13.778931 26668 consensus_queue.cc:1035] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cc7c218c8484c2eb25c6fd40cd54623" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:13.779047 26662 consensus_queue.cc:1035] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f68a003b78b54c65b1b5c4452cf12b85" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 45101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:13.782049 26673 mvcc.cc:205] Tried to move back new op lower bound from 7316091673715060736 to 7316091673649913856. Current Snapshot: MvccSnapshot[applied={T|T < 7316091673715060736 or (T in {7316091673715060736})}]
I20260808 02:13:13.784193 25947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:13.784309 25947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:13.785480 25947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94d3c9a87d75410f860fa3cd5966f4c4: Sending DeleteTablet for 3 replicas of tablet 488f26973aa449d0ad9c2d6520d46d9e
I20260808 02:13:13.785849 26137 tablet_service.cc:1514] Processing DeleteTablet for tablet 488f26973aa449d0ad9c2d6520d46d9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:13 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:48340
I20260808 02:13:13.785959 26325 tablet_service.cc:1514] Processing DeleteTablet for tablet 488f26973aa449d0ad9c2d6520d46d9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:13 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:57258
I20260808 02:13:13.786382 26528 tablet_service.cc:1514] Processing DeleteTablet for tablet 488f26973aa449d0ad9c2d6520d46d9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:13 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:34526
I20260808 02:13:13.786757 26678 mvcc.cc:205] Tried to move back new op lower bound from 7316091673715060736 to 7316091673649913856. Current Snapshot: MvccSnapshot[applied={T|T < 7316091673715060736}]
I20260808 02:13:13.787101 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 26039
I20260808 02:13:13.791816 26685 tablet_replica.cc:330] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: stopping tablet replica
I20260808 02:13:13.791972 26685 raft_consensus.cc:2229] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:13.792074 26685 raft_consensus.cc:2258] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:13.792271 26688 tablet_replica.cc:330] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: stopping tablet replica
I20260808 02:13:13.792379 26688 raft_consensus.cc:2229] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:13.792557 26688 raft_consensus.cc:2258] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:13.794117 26688 ts_tablet_manager.cc:1905] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:13.795917 26688 ts_tablet_manager.cc:1918] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:13.796041 26688 log.cc:1196] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/488f26973aa449d0ad9c2d6520d46d9e
I20260808 02:13:13.796393 26688 ts_tablet_manager.cc:1939] T 488f26973aa449d0ad9c2d6520d46d9e P 69d669b25b784875852e7036dc72a30f: Deleting consensus metadata
I20260808 02:13:13.797448 26685 ts_tablet_manager.cc:1905] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:13.798921 26685 ts_tablet_manager.cc:1918] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:13.798988 26685 log.cc:1196] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/488f26973aa449d0ad9c2d6520d46d9e
I20260808 02:13:13.799297 26685 ts_tablet_manager.cc:1939] T 488f26973aa449d0ad9c2d6520d46d9e P 4cc7c218c8484c2eb25c6fd40cd54623: Deleting consensus metadata
I20260808 02:13:13.799844 25932 catalog_manager.cc:4841] TS 4cc7c218c8484c2eb25c6fd40cd54623 (127.19.58.194:41721): tablet 488f26973aa449d0ad9c2d6520d46d9e (table test-table [id=bdc90c1431c24adc9490edeb78c4a952]) successfully deleted
I20260808 02:13:13.803872 25930 catalog_manager.cc:4841] TS 69d669b25b784875852e7036dc72a30f (127.19.58.195:45431): tablet 488f26973aa449d0ad9c2d6520d46d9e (table test-table [id=bdc90c1431c24adc9490edeb78c4a952]) successfully deleted
I20260808 02:13:13.804430 26325 tablet_service.cc:1514] Processing DeleteTablet for tablet 488f26973aa449d0ad9c2d6520d46d9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:13 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:57258
W20260808 02:13:13.804805 25932 catalog_manager.cc:4808] TS 4cc7c218c8484c2eb25c6fd40cd54623 (127.19.58.194:41721): delete failed for tablet 488f26973aa449d0ad9c2d6520d46d9e because the tablet was not found. No further retry: Not found: Tablet not found: 488f26973aa449d0ad9c2d6520d46d9e
W20260808 02:13:13.805806 25930 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:45101 (error 108)
I20260808 02:13:13.806000 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 26240
I20260808 02:13:13.813266 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.813093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:45101 (local address 127.0.0.1:48350)
0808 02:13:13.813190 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:13.813228 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:45101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260808 02:13:13.813496 25930 catalog_manager.cc:4584] TS f68a003b78b54c65b1b5c4452cf12b85 (127.19.58.193:45101): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 488f26973aa449d0ad9c2d6520d46d9e: Network error: Client connection negotiation failed: client connection to 127.19.58.193:45101: connect: Connection refused (error 111)
I20260808 02:13:13.814028 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 26434
I20260808 02:13:13.826112 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 25911
2026-08-08T02:13:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2794 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T02:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:13Z Disabled control of system clock
I20260808 02:13:13.862156 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:41163
--webserver_interface=127.19.58.254
--webserver_port=0
--builtin_ntp_servers=127.19.58.212:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:41163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:13:13.940621 26696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:13.940809 26696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:13.940827 26696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:13.941915 26696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:13.941949 26696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:13.941960 26696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:13.941972 26696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:13.941987 26696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:13.943244 26696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:41163
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:41163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:13.943441 26696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:13.943681 26696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:13:13.948261 26696 server_base.cc:1034] running on GCE node
W20260808 02:13:13.948228 26701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.948321 26708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:13.948228 26704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:13.948819 26696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:13.949133 26696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:13.950275 26696 hybrid_clock.cc:648] HybridClock initialized: now 1786155193950255 us; error 30 us; skew 500 ppm
I20260808 02:13:13.951453 26696 webserver.cc:460] Webserver started at http://127.19.58.254:35521/ using document root <none> and password file <none>
I20260808 02:13:13.951675 26696 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:13.951758 26696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:13.951889 26696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:13.952867 26696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "0fabbe8f94494414980b9e9de22e2c74"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "4088de9d444098c1b3b5f34e7fda7bd1"
server_key_iv: "79a878690622afc8ecf03d270bdf0564"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.953186 26696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "0fabbe8f94494414980b9e9de22e2c74"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "4088de9d444098c1b3b5f34e7fda7bd1"
server_key_iv: "79a878690622afc8ecf03d270bdf0564"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.954396 26696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:13:13.955091 26718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:13.955242 26696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:13.955333 26696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "0fabbe8f94494414980b9e9de22e2c74"
format_stamp: "Formatted at 2026-08-08 02:13:13 on dist-test-slave-1l3l"
server_key: "4088de9d444098c1b3b5f34e7fda7bd1"
server_key_iv: "79a878690622afc8ecf03d270bdf0564"
server_key_version: "encryptionkey@0"
I20260808 02:13:13.955418 26696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:13.958563 26696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:13.958796 26696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:13.958923 26696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:13.962670 26696 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:41163
I20260808 02:13:13.962703 26800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:41163 every 8 connection(s)
I20260808 02:13:13.962981 26696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:13.963568 26801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:13.965755 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 26696
I20260808 02:13:13.965860 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:13.966018 19691 external_mini_cluster.cc:1428] Setting key 6aa2f4b76e6ab2eb999fd96455f051fb
I20260808 02:13:13.966372 26801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74: Bootstrap starting.
I20260808 02:13:13.967027 26801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:13.967360 26801 log.cc:824] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:13.968118 26801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74: No bootstrap required, opened a new log
I20260808 02:13:13.968591 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:13.966465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57444 (local address 127.19.58.254:41163)
0808 02:13:13.966643 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:13.966651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:13.966687 (+ 36us) server_negotiation.cc:407] Connection header received
0808 02:13:13.967225 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:13.967235 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:13.967309 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:13.967459 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:13.968158 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:13.968162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:13.968162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:13.968167 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:13.968223 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:13.968238 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:13.968333 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:13.968363 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 02:13:13.968414 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260808 02:13:13.969745 26801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fabbe8f94494414980b9e9de22e2c74" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41163 } }
I20260808 02:13:13.969871 26801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:13.969893 26801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fabbe8f94494414980b9e9de22e2c74, State: Initialized, Role: FOLLOWER
I20260808 02:13:13.970002 26801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fabbe8f94494414980b9e9de22e2c74" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41163 } }
I20260808 02:13:13.970333 26801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:13.970360 26801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:13.970387 26801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:13.970881 26801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fabbe8f94494414980b9e9de22e2c74" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41163 } }
I20260808 02:13:13.970971 26801 leader_election.cc:302] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fabbe8f94494414980b9e9de22e2c74; no voters:
I20260808 02:13:13.971117 26801 leader_election.cc:288] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:13.971170 26807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:13.971329 26807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [term 1 LEADER]: Becoming Leader. State: Replica: 0fabbe8f94494414980b9e9de22e2c74, State: Running, Role: LEADER
I20260808 02:13:13.971375 26801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:13.971483 26807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fabbe8f94494414980b9e9de22e2c74" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41163 } }
I20260808 02:13:13.971863 26809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fabbe8f94494414980b9e9de22e2c74. Latest consensus state: current_term: 1 leader_uuid: "0fabbe8f94494414980b9e9de22e2c74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fabbe8f94494414980b9e9de22e2c74" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41163 } } }
I20260808 02:13:13.971959 26809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:13.972147 26808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fabbe8f94494414980b9e9de22e2c74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fabbe8f94494414980b9e9de22e2c74" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41163 } } }
I20260808 02:13:13.972189 26827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:13.972220 26808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:13.972620 26827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:13.974184 26827 catalog_manager.cc:1347] Generated new cluster ID: e2e7c052b2254d63a76d87d28bb4e9ec
I20260808 02:13:13.974228 26827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:13.989939 26827 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:13.990095 26827 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:13.995870 26827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74: Generated new TSK 0
I20260808 02:13:13.996068 26827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:13.998857 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41163
--builtin_ntp_servers=127.19.58.212:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:13:14.102077 26840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:14.102265 26840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:14.102286 26840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:14.103402 26840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:14.103441 26840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:14.103466 26840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:14.104784 26840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:14.105010 26840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:14.105281 26840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:14.110936 26851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:14.110942 26853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:14.110953 26855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:14.111212 26840 server_base.cc:1034] running on GCE node
I20260808 02:13:14.111404 26840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:14.111620 26840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:14.112756 26840 hybrid_clock.cc:648] HybridClock initialized: now 1786155194112747 us; error 51 us; skew 500 ppm
I20260808 02:13:14.113991 26840 webserver.cc:460] Webserver started at http://127.19.58.193:40209/ using document root <none> and password file <none>
I20260808 02:13:14.114195 26840 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:14.114243 26840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:14.114365 26840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:14.115371 26840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "3be7bd49b89144e28bb823973dbe5e7f"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "bd7a612876c37cc413232be22eaeba9d"
server_key_iv: "3f720f50f7e225cc514360cac2f1b28b"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.115729 26840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "3be7bd49b89144e28bb823973dbe5e7f"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "bd7a612876c37cc413232be22eaeba9d"
server_key_iv: "3f720f50f7e225cc514360cac2f1b28b"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.116951 26840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:14.117645 26865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.117805 26840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:14.117910 26840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "3be7bd49b89144e28bb823973dbe5e7f"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "bd7a612876c37cc413232be22eaeba9d"
server_key_iv: "3f720f50f7e225cc514360cac2f1b28b"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.117980 26840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:14.121438 26840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:14.121641 26840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:14.121733 26840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:14.121893 26840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:14.122138 26840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:14.122169 26840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.122187 26840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:14.122205 26840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.127535 26840 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:34787
I20260808 02:13:14.127592 27034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:34787 every 8 connection(s)
I20260808 02:13:14.127885 26840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:14.130553 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.128484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:55391 (local address 127.19.58.254:41163)
0808 02:13:14.128527 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.128529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.128741 (+ 212us) server_negotiation.cc:407] Connection header received
0808 02:13:14.129185 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.129187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.129194 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.129246 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.130317 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.130318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.130319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.130324 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.130338 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.130342 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.130392 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.130472 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.130485 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:13:14.130728 27038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.128460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:41163 (local address 127.19.58.193:55391)
0808 02:13:14.128618 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:14.128659 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:14.129096 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:14.129307 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:14.129311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:14.129320 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:14.130178 (+ 858us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:14.130179 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:14.130204 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:14.130230 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:14.130402 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:14.130408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:14.130411 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:14.130479 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:13:14.130536 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260808 02:13:14.131165 27036 heartbeater.cc:347] Connected to a master server at 127.19.58.254:41163
I20260808 02:13:14.131240 27036 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:14.131377 27036 heartbeater.cc:511] Master 127.19.58.254:41163 requested a full tablet report, sending...
I20260808 02:13:14.131851 26739 ts_manager.cc:194] Registered new tserver with Master: 3be7bd49b89144e28bb823973dbe5e7f (127.19.58.193:34787)
I20260808 02:13:14.134935 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 26840
I20260808 02:13:14.135025 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:14.135178 19691 external_mini_cluster.cc:1428] Setting key 97504b025ce956ee390901c8048490b7
I20260808 02:13:14.136013 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41163
--builtin_ntp_servers=127.19.58.212:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:13:14.216566 27039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:14.216754 27039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:14.216775 27039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:14.217900 27039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:14.217938 27039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:14.217964 27039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:14.219267 27039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:14.219484 27039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:14.219789 27039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:14.224563 27051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:14.224576 27055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:14.224611 27047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:14.224741 27039 server_base.cc:1034] running on GCE node
I20260808 02:13:14.224949 27039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:14.225144 27039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:14.226290 27039 hybrid_clock.cc:648] HybridClock initialized: now 1786155194226255 us; error 47 us; skew 500 ppm
I20260808 02:13:14.227489 27039 webserver.cc:460] Webserver started at http://127.19.58.194:39293/ using document root <none> and password file <none>
I20260808 02:13:14.227685 27039 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:14.227770 27039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:14.227905 27039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:14.228886 27039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "795e93f0ce0b486aa366772dc179c45e"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "40f22fe9a52e9287dc7147a07315bee6"
server_key_iv: "faa66ff1f8d71bb456b4dc0e74f7aa06"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.229214 27039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "795e93f0ce0b486aa366772dc179c45e"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "40f22fe9a52e9287dc7147a07315bee6"
server_key_iv: "faa66ff1f8d71bb456b4dc0e74f7aa06"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.230425 27039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:14.231081 27064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.231227 27039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:14.231314 27039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "795e93f0ce0b486aa366772dc179c45e"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "40f22fe9a52e9287dc7147a07315bee6"
server_key_iv: "faa66ff1f8d71bb456b4dc0e74f7aa06"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.231379 27039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:14.234593 27039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:14.234838 27039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:14.234959 27039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:14.235144 27039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:14.235442 27039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:14.235477 27039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.235502 27039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:14.235528 27039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.240923 27039 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:42253
I20260808 02:13:14.240990 27229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:42253 every 8 connection(s)
I20260808 02:13:14.241261 27039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:14.243805 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.241877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:49509 (local address 127.19.58.254:41163)
0808 02:13:14.241942 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.241945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.242123 (+ 178us) server_negotiation.cc:407] Connection header received
0808 02:13:14.242545 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.242546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.242551 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.242590 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.243586 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.243587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.243587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.243591 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.243604 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.243608 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.243650 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.243731 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.243745 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:13:14.243988 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.241851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:41163 (local address 127.19.58.194:49509)
0808 02:13:14.242020 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:14.242050 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:14.242454 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:14.242638 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:14.242641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:14.242649 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:14.243460 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:14.243460 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:14.243484 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:14.243510 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:14.243656 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:14.243659 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:14.243661 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:13:14.243734 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 02:13:14.243810 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260808 02:13:14.244398 27231 heartbeater.cc:347] Connected to a master server at 127.19.58.254:41163
I20260808 02:13:14.244477 27231 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:14.244609 27231 heartbeater.cc:511] Master 127.19.58.254:41163 requested a full tablet report, sending...
I20260808 02:13:14.244884 26739 ts_manager.cc:194] Registered new tserver with Master: 795e93f0ce0b486aa366772dc179c45e (127.19.58.194:42253)
I20260808 02:13:14.251000 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 27039
I20260808 02:13:14.251077 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:14.251224 19691 external_mini_cluster.cc:1428] Setting key 6ad805c38f04b8adf65b6d8a593f94cc
I20260808 02:13:14.252063 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41163
--builtin_ntp_servers=127.19.58.212:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 02:13:14.331941 27237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:14.332137 27237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:14.332158 27237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:14.333283 27237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:14.333330 27237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:14.333355 27237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:14.334780 27237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:14.335032 27237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:14.335345 27237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:14.340106 27248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:14.340106 27243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:14.340165 27244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:14.340220 27237 server_base.cc:1034] running on GCE node
I20260808 02:13:14.340498 27237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:14.340709 27237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:14.341843 27237 hybrid_clock.cc:648] HybridClock initialized: now 1786155194341827 us; error 34 us; skew 500 ppm
I20260808 02:13:14.343079 27237 webserver.cc:460] Webserver started at http://127.19.58.195:38287/ using document root <none> and password file <none>
I20260808 02:13:14.343299 27237 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:14.343348 27237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:14.343477 27237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:14.344506 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "d8c4fa111e0a440a955fa809b904f398"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "aa02b2b10e38f72a4cbccf36f57bf729"
server_key_iv: "9ad91730cf4759007e1573e2e5ebf62c"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.344832 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "d8c4fa111e0a440a955fa809b904f398"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "aa02b2b10e38f72a4cbccf36f57bf729"
server_key_iv: "9ad91730cf4759007e1573e2e5ebf62c"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.346033 27237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:13:14.346742 27257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.346885 27237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:14.346971 27237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "d8c4fa111e0a440a955fa809b904f398"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "aa02b2b10e38f72a4cbccf36f57bf729"
server_key_iv: "9ad91730cf4759007e1573e2e5ebf62c"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.347034 27237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:14.350208 27237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:14.350466 27237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:14.350591 27237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:14.350767 27237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:14.351046 27237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:14.351079 27237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.351105 27237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:14.351130 27237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.356542 27237 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:40749
I20260808 02:13:14.356575 27424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:40749 every 8 connection(s)
I20260808 02:13:14.356846 27237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:14.359578 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.357493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:60301 (local address 127.19.58.254:41163)
0808 02:13:14.357547 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.357549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.357820 (+ 271us) server_negotiation.cc:407] Connection header received
0808 02:13:14.358261 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.358261 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.358266 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.358304 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.359389 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.359390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.359390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.359394 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.359412 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.359416 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.359454 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.359515 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.359526 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 02:13:14.359762 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.357481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:41163 (local address 127.19.58.195:60301)
0808 02:13:14.357718 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:14.357755 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:14.358168 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:14.358362 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:14.358366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:14.358374 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:14.359256 (+ 882us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:14.359256 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:14.359281 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:14.359307 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:14.359461 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:14.359465 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:14.359467 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:13:14.359518 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 02:13:14.359563 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":80,"threads_started":1}
I20260808 02:13:14.360175 27425 heartbeater.cc:347] Connected to a master server at 127.19.58.254:41163
I20260808 02:13:14.360255 27425 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:14.360364 27425 heartbeater.cc:511] Master 127.19.58.254:41163 requested a full tablet report, sending...
I20260808 02:13:14.360677 26739 ts_manager.cc:194] Registered new tserver with Master: d8c4fa111e0a440a955fa809b904f398 (127.19.58.195:40749)
I20260808 02:13:14.366964 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 27237
I20260808 02:13:14.367058 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:14.367244 19691 external_mini_cluster.cc:1428] Setting key 8028989b2412dd006696e51cdf51dd03
I20260808 02:13:14.368095 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:13:14.370625 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.369777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57460 (local address 127.19.58.254:41163)
0808 02:13:14.369818 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.369820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.369864 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:13:14.369920 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.369920 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.369925 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.369967 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.370491 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.370492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.370492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.370494 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.370502 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.370504 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.370533 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.370567 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.370577 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:13:14.372082 26739 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:14.373613 26739 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:14.377249 27433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.375529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:42253 (local address 127.0.0.1:60074)
0808 02:13:14.375633 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:14.375672 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:14.375752 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:14.376345 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:14.376345 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:14.376347 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:14.376801 (+ 454us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:14.376802 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:14.376817 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:14.376860 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:14.377098 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:14.377104 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:14.377107 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:14.377167 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 02:13:14.377181 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":55,"threads_started":1}
I20260808 02:13:14.377270 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.375628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60074 (local address 127.19.58.194:42253)
0808 02:13:14.376201 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.376205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.376214 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:14.376237 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.376239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.376248 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.376474 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.376952 (+ 478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.376959 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.376960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.376963 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.376995 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.377016 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.377099 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.377172 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.377185 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":146,"threads_started":1}
I20260808 02:13:14.377568 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.375857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49370 (local address 127.19.58.193:34787)
0808 02:13:14.376031 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.376034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.376079 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:13:14.376137 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.376139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.376148 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.376501 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.377105 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.377112 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.377112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.377117 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.377152 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.377182 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.377452 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.377473 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.377487 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260808 02:13:14.377609 27440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.375846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51528 (local address 127.19.58.195:40749)
0808 02:13:14.375997 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.376002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.376079 (+ 77us) server_negotiation.cc:407] Connection header received
0808 02:13:14.376144 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.376146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.376153 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.376223 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.377257 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.377260 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.377261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.377264 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.377288 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.377302 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.377351 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.377526 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.377540 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260808 02:13:14.377372 27439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.375723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:34787 (local address 127.0.0.1:49370)
0808 02:13:14.376026 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:14.376034 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:14.376086 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:14.376319 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:14.376320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:14.376322 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:14.377015 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:14.377016 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:14.377020 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:14.377029 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:14.377259 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:14.377260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:14.377261 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 02:13:14.377290 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 02:13:14.377303 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":63,"threads_started":1}
I20260808 02:13:14.377476 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.375529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:40749 (local address 127.0.0.1:51528)
0808 02:13:14.376018 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:14.376028 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:14.376087 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:14.376225 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:14.376231 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:14.376236 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:14.376860 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:14.376861 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:14.376867 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:14.376880 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:14.377374 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:14.377374 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:14.377375 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 02:13:14.377414 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 02:13:14.377425 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":3,"thread_start_us":68,"threads_started":1}
I20260808 02:13:14.378239 27132 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2967f2f7654994a84316b10e55bf81 (DEFAULT_TABLE table=test-table [id=3aa2e385f6df46b09885f7307ae2141b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:14.378382 26936 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2967f2f7654994a84316b10e55bf81 (DEFAULT_TABLE table=test-table [id=3aa2e385f6df46b09885f7307ae2141b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:14.378439 27330 tablet_service.cc:1467] Processing CreateTablet for tablet 3b2967f2f7654994a84316b10e55bf81 (DEFAULT_TABLE table=test-table [id=3aa2e385f6df46b09885f7307ae2141b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:14.378636 27132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2967f2f7654994a84316b10e55bf81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:14.378727 27330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2967f2f7654994a84316b10e55bf81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:14.378721 26936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b2967f2f7654994a84316b10e55bf81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:14.381192 27445 tablet_bootstrap.cc:492] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f: Bootstrap starting.
I20260808 02:13:14.381752 27446 tablet_bootstrap.cc:492] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: Bootstrap starting.
I20260808 02:13:14.381923 27445 tablet_bootstrap.cc:654] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:14.381948 27447 tablet_bootstrap.cc:492] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: Bootstrap starting.
I20260808 02:13:14.382339 27445 log.cc:824] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:14.382368 27446 tablet_bootstrap.cc:654] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:14.382488 27447 tablet_bootstrap.cc:654] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:14.382651 27446 log.cc:824] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:14.382759 27447 log.cc:824] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:14.383327 27446 tablet_bootstrap.cc:492] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: No bootstrap required, opened a new log
I20260808 02:13:14.383386 27446 ts_tablet_manager.cc:1397] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:14.383464 27447 tablet_bootstrap.cc:492] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: No bootstrap required, opened a new log
I20260808 02:13:14.383464 27445 tablet_bootstrap.cc:492] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f: No bootstrap required, opened a new log
I20260808 02:13:14.383533 27447 ts_tablet_manager.cc:1397] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:14.383533 27445 ts_tablet_manager.cc:1397] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:14.385339 27446 raft_consensus.cc:348] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 } } peers { permanent_uuid: "d8c4fa111e0a440a955fa809b904f398" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 40749 } } peers { permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 } }
I20260808 02:13:14.385466 27446 raft_consensus.cc:374] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:14.385497 27446 raft_consensus.cc:729] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c4fa111e0a440a955fa809b904f398, State: Initialized, Role: FOLLOWER
I20260808 02:13:14.385622 27446 consensus_queue.cc:260] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 } } peers { permanent_uuid: "d8c4fa111e0a440a955fa809b904f398" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 40749 } } peers { permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 } }
I20260808 02:13:14.385859 27446 ts_tablet_manager.cc:1428] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:13:14.385876 27425 heartbeater.cc:503] Master 127.19.58.254:41163 was elected leader, sending a full tablet report...
I20260808 02:13:14.385970 27445 raft_consensus.cc:348] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 } } peers { permanent_uuid: "d8c4fa111e0a440a955fa809b904f398" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 40749 } } peers { permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 } }
I20260808 02:13:14.385970 27447 raft_consensus.cc:348] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 } } peers { permanent_uuid: "d8c4fa111e0a440a955fa809b904f398" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 40749 } } peers { permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 } }
I20260808 02:13:14.386119 27447 raft_consensus.cc:374] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:14.386119 27445 raft_consensus.cc:374] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:14.386159 27447 raft_consensus.cc:729] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795e93f0ce0b486aa366772dc179c45e, State: Initialized, Role: FOLLOWER
I20260808 02:13:14.386159 27445 raft_consensus.cc:729] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3be7bd49b89144e28bb823973dbe5e7f, State: Initialized, Role: FOLLOWER
I20260808 02:13:14.386276 27445 consensus_queue.cc:260] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 } } peers { permanent_uuid: "d8c4fa111e0a440a955fa809b904f398" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 40749 } } peers { permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 } }
I20260808 02:13:14.386267 27447 consensus_queue.cc:260] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 } } peers { permanent_uuid: "d8c4fa111e0a440a955fa809b904f398" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 40749 } } peers { permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 } }
I20260808 02:13:14.386520 27447 ts_tablet_manager.cc:1428] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:14.386520 27445 ts_tablet_manager.cc:1428] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:13:14.386601 27036 heartbeater.cc:503] Master 127.19.58.254:41163 was elected leader, sending a full tablet report...
I20260808 02:13:14.386637 27231 heartbeater.cc:503] Master 127.19.58.254:41163 was elected leader, sending a full tablet report...
W20260808 02:13:14.491838 27235 tablet.cc:2369] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:14.511955 27459 raft_consensus.cc:482] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:14.512094 27459 raft_consensus.cc:504] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 } } peers { permanent_uuid: "d8c4fa111e0a440a955fa809b904f398" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 40749 } } peers { permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 } }
I20260808 02:13:14.512503 27459 leader_election.cc:288] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 795e93f0ce0b486aa366772dc179c45e (127.19.58.194:42253), 3be7bd49b89144e28bb823973dbe5e7f (127.19.58.193:34787)
I20260808 02:13:14.513775 27427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.512694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:34787 (local address 127.19.58.195:52525)
0808 02:13:14.512750 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:14.512764 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:14.512822 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:14.513047 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:14.513048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:14.513051 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:14.513470 (+ 419us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:14.513471 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:14.513479 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:14.513490 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:14.513658 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:14.513658 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:14.513659 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 02:13:14.513706 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 02:13:14.513719 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 02:13:14.513797 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.512850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:52525 (local address 127.19.58.193:34787)
0808 02:13:14.512931 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.512933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.512942 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:14.512971 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.512972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.512977 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.513031 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.513573 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.513574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.513574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.513579 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.513599 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.513604 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.513649 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.513714 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.513729 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 02:13:14.513979 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.512796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:45457 (local address 127.19.58.194:42253)
0808 02:13:14.512851 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.512853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.512965 (+ 112us) server_negotiation.cc:407] Connection header received
0808 02:13:14.513010 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.513012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.513017 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.513074 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.513730 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.513731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.513732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.513736 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.513752 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.513757 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.513810 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.513901 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.513916 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 02:13:14.514158 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.512693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:42253 (local address 127.19.58.195:45457)
0808 02:13:14.512889 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:14.512898 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:14.512960 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:14.513085 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:14.513086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:14.513088 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:14.513632 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:14.513632 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:14.513639 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:14.513650 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:14.513813 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:14.513814 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:14.513814 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 02:13:14.514063 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 02:13:14.514077 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":14,"thread_start_us":113,"threads_started":1}
I20260808 02:13:14.514147 26962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2967f2f7654994a84316b10e55bf81" candidate_uuid: "d8c4fa111e0a440a955fa809b904f398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3be7bd49b89144e28bb823973dbe5e7f" is_pre_election: true
I20260808 02:13:14.514299 26962 raft_consensus.cc:2454] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c4fa111e0a440a955fa809b904f398 in term 0.
I20260808 02:13:14.514518 27163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2967f2f7654994a84316b10e55bf81" candidate_uuid: "d8c4fa111e0a440a955fa809b904f398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "795e93f0ce0b486aa366772dc179c45e" is_pre_election: true
I20260808 02:13:14.514587 27259 leader_election.cc:302] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3be7bd49b89144e28bb823973dbe5e7f, d8c4fa111e0a440a955fa809b904f398; no voters:
I20260808 02:13:14.514654 27163 raft_consensus.cc:2454] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8c4fa111e0a440a955fa809b904f398 in term 0.
I20260808 02:13:14.514719 27459 raft_consensus.cc:2781] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:14.514770 27459 raft_consensus.cc:482] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:14.514791 27459 raft_consensus.cc:3037] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:14.515543 27459 raft_consensus.cc:504] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 } } peers { permanent_uuid: "d8c4fa111e0a440a955fa809b904f398" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 40749 } } peers { permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 } }
I20260808 02:13:14.515658 27459 leader_election.cc:288] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [CANDIDATE]: Term 1 election: Requested vote from peers 795e93f0ce0b486aa366772dc179c45e (127.19.58.194:42253), 3be7bd49b89144e28bb823973dbe5e7f (127.19.58.193:34787)
I20260808 02:13:14.515797 27163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2967f2f7654994a84316b10e55bf81" candidate_uuid: "d8c4fa111e0a440a955fa809b904f398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "795e93f0ce0b486aa366772dc179c45e"
I20260808 02:13:14.515873 27163 raft_consensus.cc:3037] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:14.515868 26962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b2967f2f7654994a84316b10e55bf81" candidate_uuid: "d8c4fa111e0a440a955fa809b904f398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3be7bd49b89144e28bb823973dbe5e7f"
I20260808 02:13:14.515945 26962 raft_consensus.cc:3037] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:14.516563 27163 raft_consensus.cc:2454] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c4fa111e0a440a955fa809b904f398 in term 1.
I20260808 02:13:14.516674 26962 raft_consensus.cc:2454] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8c4fa111e0a440a955fa809b904f398 in term 1.
I20260808 02:13:14.516728 27262 leader_election.cc:302] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 795e93f0ce0b486aa366772dc179c45e, d8c4fa111e0a440a955fa809b904f398; no voters:
I20260808 02:13:14.516813 27459 raft_consensus.cc:2781] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:14.516960 27459 raft_consensus.cc:686] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 1 LEADER]: Becoming Leader. State: Replica: d8c4fa111e0a440a955fa809b904f398, State: Running, Role: LEADER
I20260808 02:13:14.517058 27459 consensus_queue.cc:237] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 } } peers { permanent_uuid: "d8c4fa111e0a440a955fa809b904f398" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 40749 } } peers { permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 } }
I20260808 02:13:14.517781 26738 catalog_manager.cc:5495] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 reported cstate change: term changed from 0 to 1, leader changed from <none> to d8c4fa111e0a440a955fa809b904f398 (127.19.58.195). New cstate: current_term: 1 leader_uuid: "d8c4fa111e0a440a955fa809b904f398" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c4fa111e0a440a955fa809b904f398" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 40749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:14.524246 27440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.523362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51542 (local address 127.19.58.195:40749)
0808 02:13:14.523435 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.523436 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.523445 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:14.523471 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.523472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.523482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.523526 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.524058 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.524059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.524059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.524063 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.524078 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.524081 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.524128 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.524176 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.524187 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 02:13:14.525681 27163 raft_consensus.cc:1261] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e [term 1 FOLLOWER]: Refusing update from remote peer d8c4fa111e0a440a955fa809b904f398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:14.525732 26962 raft_consensus.cc:1261] T 3b2967f2f7654994a84316b10e55bf81 P 3be7bd49b89144e28bb823973dbe5e7f [term 1 FOLLOWER]: Refusing update from remote peer d8c4fa111e0a440a955fa809b904f398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:14.525967 27459 consensus_queue.cc:1035] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "795e93f0ce0b486aa366772dc179c45e" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:14.526088 27467 consensus_queue.cc:1035] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3be7bd49b89144e28bb823973dbe5e7f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:14.530135 27472 mvcc.cc:205] Tried to move back new op lower bound from 7316091676775157760 to 7316091676742438912. Current Snapshot: MvccSnapshot[applied={T|T < 7316091676775157760}]
I20260808 02:13:14.530320 26738 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:14.530402 26738 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:14.531008 26738 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fabbe8f94494414980b9e9de22e2c74: Sending DeleteTablet for 3 replicas of tablet 3b2967f2f7654994a84316b10e55bf81
I20260808 02:13:14.531334 27132 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2967f2f7654994a84316b10e55bf81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:14 UTC) from {username='slave'} at 127.0.0.1:60074
I20260808 02:13:14.531345 26936 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2967f2f7654994a84316b10e55bf81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:14 UTC) from {username='slave'} at 127.0.0.1:49370
I20260808 02:13:14.531543 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 26840
I20260808 02:13:14.531618 27330 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b2967f2f7654994a84316b10e55bf81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:14 UTC) from {username='slave'} at 127.0.0.1:51528
I20260808 02:13:14.537544 27483 tablet_replica.cc:330] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: stopping tablet replica
I20260808 02:13:14.537644 27483 raft_consensus.cc:2229] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:14.537738 27483 raft_consensus.cc:2258] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:14.539160 27483 ts_tablet_manager.cc:1905] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:14.540740 27483 ts_tablet_manager.cc:1918] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:14.540817 27483 log.cc:1196] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/3b2967f2f7654994a84316b10e55bf81
I20260808 02:13:14.541080 27483 ts_tablet_manager.cc:1939] T 3b2967f2f7654994a84316b10e55bf81 P 795e93f0ce0b486aa366772dc179c45e: Deleting consensus metadata
I20260808 02:13:14.541471 26721 catalog_manager.cc:4841] TS 795e93f0ce0b486aa366772dc179c45e (127.19.58.194:42253): tablet 3b2967f2f7654994a84316b10e55bf81 (table test-table [id=3aa2e385f6df46b09885f7307ae2141b]) successfully deleted
I20260808 02:13:14.545965 27485 tablet_replica.cc:330] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: stopping tablet replica
I20260808 02:13:14.546132 27485 raft_consensus.cc:2229] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:14.546321 27485 raft_consensus.cc:2258] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:14.546967 27485 ts_tablet_manager.cc:1905] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:14.549190 27485 ts_tablet_manager.cc:1918] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:14.549309 27485 log.cc:1196] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/3b2967f2f7654994a84316b10e55bf81
I20260808 02:13:14.549618 27485 ts_tablet_manager.cc:1939] T 3b2967f2f7654994a84316b10e55bf81 P d8c4fa111e0a440a955fa809b904f398: Deleting consensus metadata
I20260808 02:13:14.550076 26720 catalog_manager.cc:4841] TS d8c4fa111e0a440a955fa809b904f398 (127.19.58.195:40749): tablet 3b2967f2f7654994a84316b10e55bf81 (table test-table [id=3aa2e385f6df46b09885f7307ae2141b]) successfully deleted
W20260808 02:13:14.553459 26720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:34787 (error 108)
I20260808 02:13:14.553915 27439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.553793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:34787 (local address 127.0.0.1:49384)
0808 02:13:14.553844 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:14.553883 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 02:13:14.554004 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 27039
W20260808 02:13:14.554072 26720 catalog_manager.cc:4584] TS 3be7bd49b89144e28bb823973dbe5e7f (127.19.58.193:34787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b2967f2f7654994a84316b10e55bf81: Network error: Client connection negotiation failed: client connection to 127.19.58.193:34787: connect: Connection refused (error 111)
I20260808 02:13:14.559558 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 27237
I20260808 02:13:14.567242 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 26696
2026-08-08T02:13:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (734 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 02:13:14.589555 19691 mini_oidc.cc:152] Starting JWKS server
I20260808 02:13:14.590767 19691 webserver.cc:460] Webserver started at https://127.0.0.1:33371/ using document root <none> and password file <none>
I20260808 02:13:14.590852 19691 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:13:14.591023 19691 webserver.cc:460] Webserver started at http://127.0.0.1:33717/ using document root <none> and password file <none>
2026-08-08T02:13:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:14Z Disabled control of system clock
I20260808 02:13:14.598668 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40597
--webserver_interface=127.19.58.254
--webserver_port=0
--builtin_ntp_servers=127.19.58.212:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:40597
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33371/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/testchainca.cert with env {}
W20260808 02:13:14.677151 27501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:14.677333 27501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:14.677353 27501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:14.678427 27501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:14.678462 27501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:14.678472 27501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:14.678483 27501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:14.678498 27501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:13:14.678508 27501 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:13:14.678524 27501 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33371/jwks.json
I20260808 02:13:14.680150 27501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:40597
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33371/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:14.680346 27501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:14.680498 27501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:14.685016 27512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:14.685109 27509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:14.685178 27510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:14.685179 27501 server_base.cc:1034] running on GCE node
I20260808 02:13:14.685497 27501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:14.685760 27501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:14.686918 27501 hybrid_clock.cc:648] HybridClock initialized: now 1786155194686904 us; error 36 us; skew 500 ppm
I20260808 02:13:14.688151 27501 webserver.cc:460] Webserver started at http://127.19.58.254:38503/ using document root <none> and password file <none>
I20260808 02:13:14.688359 27501 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:14.688406 27501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:14.688527 27501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:14.689489 27501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "5561241828164613b659c7b119498136"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "334273f3302b7ebfd6093a68be6633c1"
server_key_iv: "991a3201639a507ac4028503d002e9ab"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.689793 27501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "5561241828164613b659c7b119498136"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "334273f3302b7ebfd6093a68be6633c1"
server_key_iv: "991a3201639a507ac4028503d002e9ab"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.690943 27501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:14.691579 27521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.691772 27501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:14.691846 27501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "5561241828164613b659c7b119498136"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "334273f3302b7ebfd6093a68be6633c1"
server_key_iv: "991a3201639a507ac4028503d002e9ab"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.691919 27501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:14.712633 27501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:14.712879 27501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:14.713001 27501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:14.716601 27501 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:40597
I20260808 02:13:14.716642 27597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:40597 every 8 connection(s)
I20260808 02:13:14.716923 27501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:14.717793 27601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:14.719952 27601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136: Bootstrap starting.
I20260808 02:13:14.720489 27601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:14.720758 27601 log.cc:824] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:14.721343 27601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136: No bootstrap required, opened a new log
I20260808 02:13:14.722586 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 27501
I20260808 02:13:14.722675 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:14.722867 19691 external_mini_cluster.cc:1428] Setting key 196859d91a015495fc231042944c19eb
I20260808 02:13:14.723060 27601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561241828164613b659c7b119498136" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40597 } }
I20260808 02:13:14.723201 27601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:14.723237 27601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5561241828164613b659c7b119498136, State: Initialized, Role: FOLLOWER
I20260808 02:13:14.723349 27601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561241828164613b659c7b119498136" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40597 } }
I20260808 02:13:14.723438 27601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:14.723481 27601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:14.723524 27601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:14.724062 27601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561241828164613b659c7b119498136" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40597 } }
I20260808 02:13:14.724149 27601 leader_election.cc:302] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5561241828164613b659c7b119498136; no voters:
I20260808 02:13:14.724291 27601 leader_election.cc:288] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:14.724335 27613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:14.724448 27613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [term 1 LEADER]: Becoming Leader. State: Replica: 5561241828164613b659c7b119498136, State: Running, Role: LEADER
I20260808 02:13:14.724538 27613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561241828164613b659c7b119498136" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40597 } }
I20260808 02:13:14.724541 27601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:14.724923 27615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5561241828164613b659c7b119498136. Latest consensus state: current_term: 1 leader_uuid: "5561241828164613b659c7b119498136" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561241828164613b659c7b119498136" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40597 } } }
I20260808 02:13:14.724929 27614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5561241828164613b659c7b119498136" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5561241828164613b659c7b119498136" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40597 } } }
I20260808 02:13:14.725011 27615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:14.725029 27614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:14.725731 27621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:14.726187 27621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:14.727612 27621 catalog_manager.cc:1347] Generated new cluster ID: 6ed507023f434ce19b787d0c4f830063
I20260808 02:13:14.727665 27621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:14.728820 27609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.723301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33064 (local address 127.19.58.254:40597)
0808 02:13:14.723539 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.723547 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.723577 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:13:14.723683 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.723688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.723772 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.723897 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.724090 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:14.725862 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:14.726930 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:14.727480 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:14.727887 (+ 407us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:14.728226 (+ 339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.728229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.728230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.728387 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.728418 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.728427 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.728483 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.728553 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.728627 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260808 02:13:14.729635 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:13:14.759023 27621 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:14.759490 27621 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:14.761837 27621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5561241828164613b659c7b119498136: Generated new TSK 0
I20260808 02:13:14.761994 27621 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 02:13:14.785204 27650 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 02:13:14.785346 27650 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 02:13:14.785483 27609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.770785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33074 (local address 127.19.58.254:40597)
0808 02:13:14.770824 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.770826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.770925 (+ 99us) server_negotiation.cc:407] Connection header received
0808 02:13:14.771022 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.771023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.771033 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.771084 (+ 51us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:13:14.771223 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:14.771672 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:14.772777 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:14.772964 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:14.773004 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:14.773199 (+ 195us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:13:14.785266 (+ 12067us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 02:13:14.785422 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33074: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 02:13:14.785537 27609 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33074: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 02:13:14.786026 27649 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:14.770699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40597 (local address 127.0.0.1:33074)
0808 02:13:14.770859 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:14.770870 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:14.770940 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:14.771089 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:14.771090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:14.771092 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:13:14.771170 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:14.771173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:14.771775 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:14.771776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:14.772704 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:14.772707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:14.773021 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:14.773021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:14.773085 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:14.773107 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:13:14.785515 (+ 12408us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 02:13:14.785710 (+ 195us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.58.254:40597: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.19.58.254:40597: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33371/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 02:13:14.786453 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 27501
2026-08-08T02:13:14Z chronyd exiting
I20260808 02:13:14.828096 19691 test_util.cc:175] -----------------------------------------------
I20260808 02:13:14.828174 19691 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786155170542577-19691-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (239 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 02:13:14.828917 19691 mini_oidc.cc:152] Starting JWKS server
I20260808 02:13:14.829586 19691 webserver.cc:460] Webserver started at https://127.0.0.1:34315/ using document root <none> and password file <none>
I20260808 02:13:14.829631 19691 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:13:14.829766 19691 webserver.cc:460] Webserver started at http://127.0.0.1:43979/ using document root <none> and password file <none>
2026-08-08T02:13:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:14Z Disabled control of system clock
I20260808 02:13:14.837564 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:43705
--webserver_interface=127.19.58.254
--webserver_port=0
--builtin_ntp_servers=127.19.58.212:43609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:43705
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34315/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/testchainca.cert with env {}
W20260808 02:13:14.915880 27658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:14.916081 27658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:14.916111 27658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:14.917256 27658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:14.917296 27658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:14.917315 27658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:14.917330 27658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:14.917346 27658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:13:14.917362 27658 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:13:14.917385 27658 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34315/jwks.json
I20260808 02:13:14.918865 27658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:43609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:43705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:43705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34315/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:14.919100 27658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:14.919292 27658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:14.923836 27664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:14.923848 27667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:14.923964 27665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:14.924010 27658 server_base.cc:1034] running on GCE node
I20260808 02:13:14.924222 27658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:14.924501 27658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:14.925638 27658 hybrid_clock.cc:648] HybridClock initialized: now 1786155194925627 us; error 26 us; skew 500 ppm
I20260808 02:13:14.926788 27658 webserver.cc:460] Webserver started at http://127.19.58.254:35759/ using document root <none> and password file <none>
I20260808 02:13:14.927000 27658 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:14.927050 27658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:14.927174 27658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:14.928231 27658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "243024323d914ad983b8f16bad008bca"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "53dec581b3c3881dc8fac6f933d8c5d2"
server_key_iv: "144e53cb7af66bb49cc3cc95ea99d7de"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.928563 27658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "243024323d914ad983b8f16bad008bca"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "53dec581b3c3881dc8fac6f933d8c5d2"
server_key_iv: "144e53cb7af66bb49cc3cc95ea99d7de"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.929780 27658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:14.930433 27676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:14.930563 27658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:14.930645 27658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "243024323d914ad983b8f16bad008bca"
format_stamp: "Formatted at 2026-08-08 02:13:14 on dist-test-slave-1l3l"
server_key: "53dec581b3c3881dc8fac6f933d8c5d2"
server_key_iv: "144e53cb7af66bb49cc3cc95ea99d7de"
server_key_version: "encryptionkey@0"
I20260808 02:13:14.930706 27658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:14.952051 27658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:14.952379 27658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:14.952524 27658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:14.956454 27658 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:43705
I20260808 02:13:14.956492 27753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:43705 every 8 connection(s)
I20260808 02:13:14.956813 27658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:14.957458 27755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:14.960220 27755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca: Bootstrap starting.
I20260808 02:13:14.960754 27755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:14.960975 27755 log.cc:824] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:14.961380 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 27658
I20260808 02:13:14.961473 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:14.961648 27755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca: No bootstrap required, opened a new log
I20260808 02:13:14.961655 19691 external_mini_cluster.cc:1428] Setting key 79f4efab99e9a237e2d0ecd319f2eff8
I20260808 02:13:14.963615 27755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243024323d914ad983b8f16bad008bca" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43705 } }
I20260808 02:13:14.963774 27755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:14.963820 27755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243024323d914ad983b8f16bad008bca, State: Initialized, Role: FOLLOWER
I20260808 02:13:14.963945 27755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243024323d914ad983b8f16bad008bca" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43705 } }
I20260808 02:13:14.964013 27755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:14.964043 27755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:14.964089 27755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:14.964620 27755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243024323d914ad983b8f16bad008bca" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43705 } }
I20260808 02:13:14.964725 27755 leader_election.cc:302] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 243024323d914ad983b8f16bad008bca; no voters:
I20260808 02:13:14.964879 27755 leader_election.cc:288] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:14.964918 27763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:14.965062 27763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [term 1 LEADER]: Becoming Leader. State: Replica: 243024323d914ad983b8f16bad008bca, State: Running, Role: LEADER
I20260808 02:13:14.965121 27755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:14.965175 27763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243024323d914ad983b8f16bad008bca" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43705 } }
I20260808 02:13:14.965457 27763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 243024323d914ad983b8f16bad008bca. Latest consensus state: current_term: 1 leader_uuid: "243024323d914ad983b8f16bad008bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243024323d914ad983b8f16bad008bca" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43705 } } }
I20260808 02:13:14.965538 27763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:14.965481 27766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "243024323d914ad983b8f16bad008bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243024323d914ad983b8f16bad008bca" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43705 } } }
I20260808 02:13:14.965653 27766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca [sys.catalog]: This master's current role is: LEADER
W20260808 02:13:14.966809 27788 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:14.966866 27788 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:14.967140 27770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:14.967330 27770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:14.968043 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:14.962110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34550 (local address 127.19.58.254:43705)
0808 02:13:14.962283 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:14.962290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:14.962314 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:13:14.962835 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:14.962842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:14.962897 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:14.963005 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:14.963205 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:14.964562 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:14.965671 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:14.966294 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:14.966391 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:14.967299 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:14.967303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:14.967303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:14.967494 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:14.967538 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:14.967553 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:14.967591 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:14.967745 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 02:13:14.967822 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260808 02:13:14.968842 27770 catalog_manager.cc:1347] Generated new cluster ID: da11a53288ec42039efb6da566c926d8
I20260808 02:13:14.968892 27770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:14.969022 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:13:14.987262 27770 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:14.987756 27770 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:14.998970 27770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 243024323d914ad983b8f16bad008bca: Generated new TSK 0
I20260808 02:13:14.999123 27770 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 02:13:15.027742 27808 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 02:13:15.027853 27808 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260808 02:13:15.028443 27803 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:15.012623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:43705 (local address 127.0.0.1:34562)
0808 02:13:15.012768 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:15.012778 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:15.012833 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:15.012982 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:15.012983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:15.012984 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:13:15.013033 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:15.013035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.013971 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.013972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:15.014544 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:15.014546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.015014 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.015015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:15.015058 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.015062 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:13:15.028227 (+ 13165us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 02:13:15.028336 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.58.254:43705: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34315/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260808 02:13:15.028699 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 27658
2026-08-08T02:13:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (244 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 02:13:15.073411 19691 mini_oidc.cc:152] Starting JWKS server
I20260808 02:13:15.074044 19691 webserver.cc:460] Webserver started at https://127.0.0.1:33899/ using document root <none> and password file <none>
I20260808 02:13:15.074084 19691 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 02:13:15.074190 19691 webserver.cc:460] Webserver started at http://127.0.0.1:42281/ using document root <none> and password file <none>
2026-08-08T02:13:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:15Z Disabled control of system clock
I20260808 02:13:15.081997 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:44379
--webserver_interface=127.19.58.254
--webserver_port=0
--builtin_ntp_servers=127.19.58.212:35149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:44379
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33899/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 02:13:15.160940 27816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:15.161140 27816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:15.161160 27816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:15.162236 27816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:15.162271 27816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:15.162282 27816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:15.162293 27816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:15.162312 27816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:13:15.162325 27816 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:13:15.162336 27816 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33899/jwks.json
I20260808 02:13:15.163612 27816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:44379
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:44379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33899/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:15.163830 27816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:15.164065 27816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:15.168531 27822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:15.168565 27827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:15.168541 27830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:15.168578 27816 server_base.cc:1034] running on GCE node
I20260808 02:13:15.168918 27816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:15.169160 27816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:15.170331 27816 hybrid_clock.cc:648] HybridClock initialized: now 1786155195170290 us; error 28 us; skew 500 ppm
I20260808 02:13:15.171504 27816 webserver.cc:460] Webserver started at http://127.19.58.254:38429/ using document root <none> and password file <none>
I20260808 02:13:15.171731 27816 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:15.171792 27816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:15.171923 27816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:15.172909 27816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "c3f29004460c454ba3bc1366106a1ea8"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-1l3l"
server_key: "c097bc4ab1e7030583711a67b5eeb4db"
server_key_iv: "13a0f59fb526c996448af8700b7a0f07"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.173228 27816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "c3f29004460c454ba3bc1366106a1ea8"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-1l3l"
server_key: "c097bc4ab1e7030583711a67b5eeb4db"
server_key_iv: "13a0f59fb526c996448af8700b7a0f07"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.174428 27816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:15.175094 27839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:15.175230 27816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:15.175326 27816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "c3f29004460c454ba3bc1366106a1ea8"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-1l3l"
server_key: "c097bc4ab1e7030583711a67b5eeb4db"
server_key_iv: "13a0f59fb526c996448af8700b7a0f07"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.175395 27816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:15.187883 27816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:15.188120 27816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:15.188249 27816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:15.192031 27816 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:44379
I20260808 02:13:15.192060 27926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:44379 every 8 connection(s)
I20260808 02:13:15.192363 27816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:15.192945 27928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:15.195518 27928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8: Bootstrap starting.
I20260808 02:13:15.195853 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 27816
I20260808 02:13:15.195933 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:15.196094 19691 external_mini_cluster.cc:1428] Setting key eabd96609bcd292fa95b304d9fc49ef1
I20260808 02:13:15.196111 27928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:15.196365 27928 log.cc:824] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:15.197082 27928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8: No bootstrap required, opened a new log
I20260808 02:13:15.198734 27928 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f29004460c454ba3bc1366106a1ea8" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44379 } }
I20260808 02:13:15.198858 27928 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:15.198899 27928 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f29004460c454ba3bc1366106a1ea8, State: Initialized, Role: FOLLOWER
I20260808 02:13:15.199020 27928 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f29004460c454ba3bc1366106a1ea8" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44379 } }
I20260808 02:13:15.199091 27928 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:15.199122 27928 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:15.199168 27928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:15.199661 27928 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f29004460c454ba3bc1366106a1ea8" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44379 } }
I20260808 02:13:15.199774 27928 leader_election.cc:302] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3f29004460c454ba3bc1366106a1ea8; no voters:
I20260808 02:13:15.199930 27928 leader_election.cc:288] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:15.199966 27935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:15.200181 27928 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:15.200402 27935 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [term 1 LEADER]: Becoming Leader. State: Replica: c3f29004460c454ba3bc1366106a1ea8, State: Running, Role: LEADER
I20260808 02:13:15.200502 27935 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f29004460c454ba3bc1366106a1ea8" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44379 } }
I20260808 02:13:15.200814 27935 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3f29004460c454ba3bc1366106a1ea8. Latest consensus state: current_term: 1 leader_uuid: "c3f29004460c454ba3bc1366106a1ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f29004460c454ba3bc1366106a1ea8" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44379 } } }
I20260808 02:13:15.200884 27935 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:15.200959 27937 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3f29004460c454ba3bc1366106a1ea8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f29004460c454ba3bc1366106a1ea8" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44379 } } }
I20260808 02:13:15.201042 27937 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:15.201192 27948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:15.201225 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:15.196506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.19.58.254:44379)
0808 02:13:15.196678 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:15.196687 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:15.196718 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:13:15.197130 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:15.197136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:15.197216 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:15.197344 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:15.197542 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.198440 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.199302 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.199689 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.199817 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.200467 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:15.200471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:15.200471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:15.200666 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:15.200709 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:15.200725 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:15.200787 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:15.200877 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 02:13:15.200952 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260808 02:13:15.201622 27948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:15.202184 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:13:15.203146 27948 catalog_manager.cc:1347] Generated new cluster ID: b5dea4009de54337958ac7683926f89b
I20260808 02:13:15.203200 27948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:15.217129 27948 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:15.217783 27948 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:15.225798 27948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3f29004460c454ba3bc1366106a1ea8: Generated new TSK 0
I20260808 02:13:15.225970 27948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:15.235649 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:15.232335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.19.58.254:44379)
0808 02:13:15.232374 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:15.232376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:15.232409 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:13:15.232490 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:15.232490 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:15.232500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:15.232551 (+ 51us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:13:15.232658 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.233154 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.234239 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.234429 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.234478 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.234631 (+ 153us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:13:15.235465 (+ 834us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33899/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 02:13:15.235586 (+ 121us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43188: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33899/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 02:13:15.235728 27932 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43188: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33899/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 02:13:15.235791 27973 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:15.232218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44379 (local address 127.0.0.1:43188)
0808 02:13:15.232345 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:15.232355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:15.232428 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:15.232552 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:15.232553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:15.232554 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:13:15.232605 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:15.232607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.233225 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.233226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:15.234157 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:15.234160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.234492 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.234493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:15.234550 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.234556 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:13:15.235554 (+ 998us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33899/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 02:13:15.235652 (+ 98us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.58.254:44379: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33899/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260808 02:13:15.235999 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 27816
E20260808 02:13:15.253669 27974 webserver.cc:533] Webserver: sslize failed:
E20260808 02:13:15.253728 27974 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T02:13:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (208 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T02:13:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:15Z Disabled control of system clock
I20260808 02:13:15.289721 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:37479
--webserver_interface=127.19.58.254
--webserver_port=0
--builtin_ntp_servers=127.19.58.212:35635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:37479
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/testchainca.cert with env {}
W20260808 02:13:15.368343 27981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:15.368541 27981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:15.368572 27981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:15.369685 27981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:15.369724 27981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:15.369741 27981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:15.369758 27981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:15.369773 27981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 02:13:15.369791 27981 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 02:13:15.369812 27981 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 02:13:15.371773 27981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:37479
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:37479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:15.372018 27981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:15.372251 27981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:15.376869 27989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:15.376943 27981 server_base.cc:1034] running on GCE node
W20260808 02:13:15.376881 27987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:15.376883 27992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:15.377252 27981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:15.377521 27981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:15.378649 27981 hybrid_clock.cc:648] HybridClock initialized: now 1786155195378642 us; error 28 us; skew 500 ppm
I20260808 02:13:15.379784 27981 webserver.cc:460] Webserver started at http://127.19.58.254:45949/ using document root <none> and password file <none>
I20260808 02:13:15.380002 27981 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:15.380049 27981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:15.380169 27981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:15.381157 27981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "3eafdb315f23475abf1d07c894680e87"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-1l3l"
server_key: "db3bc37d362fcb160b80c44bce587516"
server_key_iv: "e6acf82d9cde9a0151167bd78dd54b5d"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.381485 27981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "3eafdb315f23475abf1d07c894680e87"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-1l3l"
server_key: "db3bc37d362fcb160b80c44bce587516"
server_key_iv: "e6acf82d9cde9a0151167bd78dd54b5d"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.382694 27981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:13:15.383311 28004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:15.383456 27981 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:15.383544 27981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "3eafdb315f23475abf1d07c894680e87"
format_stamp: "Formatted at 2026-08-08 02:13:15 on dist-test-slave-1l3l"
server_key: "db3bc37d362fcb160b80c44bce587516"
server_key_iv: "e6acf82d9cde9a0151167bd78dd54b5d"
server_key_version: "encryptionkey@0"
I20260808 02:13:15.383598 27981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:15.397221 27981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:15.397431 27981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:15.397531 27981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:15.401132 27981 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:37479
I20260808 02:13:15.401165 28088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:37479 every 8 connection(s)
I20260808 02:13:15.401434 27981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:15.402315 28089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:15.403533 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 27981
I20260808 02:13:15.403615 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:15.403798 19691 external_mini_cluster.cc:1428] Setting key f111e9571c05e13c21aaee61e4725f3c
I20260808 02:13:15.404827 28089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87: Bootstrap starting.
I20260808 02:13:15.405522 28089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:15.405827 28089 log.cc:824] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:15.406509 28089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87: No bootstrap required, opened a new log
I20260808 02:13:15.408546 28089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eafdb315f23475abf1d07c894680e87" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37479 } }
I20260808 02:13:15.408695 28089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:15.408739 28089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eafdb315f23475abf1d07c894680e87, State: Initialized, Role: FOLLOWER
I20260808 02:13:15.408856 28089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eafdb315f23475abf1d07c894680e87" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37479 } }
I20260808 02:13:15.408944 28089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:15.408984 28089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:15.409032 28089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:15.409425 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:15.404229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.19.58.254:37479)
0808 02:13:15.404456 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:15.404464 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:15.404499 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:13:15.404977 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:15.404985 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:15.405059 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:15.405212 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:15.405352 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.406597 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.407337 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.407988 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.408125 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.408645 (+ 520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:15.408649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:15.408649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:15.408836 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:15.408876 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:15.408890 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:15.408965 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:15.409049 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 02:13:15.409224 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260808 02:13:15.409660 28089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eafdb315f23475abf1d07c894680e87" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37479 } }
I20260808 02:13:15.409759 28089 leader_election.cc:302] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eafdb315f23475abf1d07c894680e87; no voters:
I20260808 02:13:15.409901 28089 leader_election.cc:288] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:15.409934 28096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:15.410048 28096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [term 1 LEADER]: Becoming Leader. State: Replica: 3eafdb315f23475abf1d07c894680e87, State: Running, Role: LEADER
I20260808 02:13:15.410126 28089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:15.410123 28096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eafdb315f23475abf1d07c894680e87" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37479 } }
I20260808 02:13:15.410248 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:13:15.410468 28096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eafdb315f23475abf1d07c894680e87. Latest consensus state: current_term: 1 leader_uuid: "3eafdb315f23475abf1d07c894680e87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eafdb315f23475abf1d07c894680e87" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37479 } } }
I20260808 02:13:15.410490 28099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3eafdb315f23475abf1d07c894680e87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eafdb315f23475abf1d07c894680e87" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37479 } } }
I20260808 02:13:15.410542 28096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:15.410567 28099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87 [sys.catalog]: This master's current role is: LEADER
W20260808 02:13:15.411765 28118 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:15.411819 28118 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:15.411868 28103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:15.412025 28103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:15.413391 28103 catalog_manager.cc:1347] Generated new cluster ID: c601f123f22144239d1a2c9eba7e20fb
I20260808 02:13:15.413448 28103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:15.437875 28103 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:15.438596 28103 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:15.447237 28103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3eafdb315f23475abf1d07c894680e87: Generated new TSK 0
I20260808 02:13:15.447407 28103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:15.485239 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:15.455034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35866 (local address 127.19.58.254:37479)
0808 02:13:15.455089 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:15.455091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:15.455190 (+ 99us) server_negotiation.cc:407] Connection header received
0808 02:13:15.455264 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:15.455265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:15.455275 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:15.455332 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0808 02:13:15.455473 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.456015 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.457100 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.457301 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.457350 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.457499 (+ 149us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 02:13:15.485045 (+ 27546us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 02:13:15.485183 (+ 138us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35866: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 02:13:15.485303 28094 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35866: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 02:13:15.485440 28134 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:15.454969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:37479 (local address 127.0.0.1:35866)
0808 02:13:15.455120 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:15.455131 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:15.455198 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:15.455345 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:15.455346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:15.455348 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 02:13:15.455417 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:15.455419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.456078 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.456079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:15.457018 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:15.457021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:15.457364 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:15.457365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:15.457421 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:15.457426 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 02:13:15.485175 (+ 27749us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 02:13:15.485325 (+ 150us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.58.254:37479: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260808 02:13:15.485828 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 27981
2026-08-08T02:13:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (225 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (86 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:15 dist-test-slave-1l3l krb5kdc[28153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:15 dist-test-slave-1l3l krb5kdc[28153](info): set up 2 sockets
Aug 08 02:13:15 dist-test-slave-1l3l krb5kdc[28153](info): commencing operation
krb5kdc: starting...
W20260808 02:13:16.765276 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:16 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:13:16.875530 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:41805
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:41805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:16.954191 28169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:16.954380 28169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:16.954407 28169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:16.955492 28169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:16.955525 28169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:16.955538 28169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:16.955549 28169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:16.955562 28169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:16.957033 28169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:41805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:41805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:16.957219 28169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:16.957383 28169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:16.961949 28179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:16.961993 28181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.961993 28169 server_base.cc:1034] running on GCE node
W20260808 02:13:16.961963 28184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:16.962317 28169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:16.962587 28169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:16.963743 28169 hybrid_clock.cc:648] HybridClock initialized: now 1786155196963720 us; error 34 us; skew 500 ppm
Aug 08 02:13:16 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:16.966243 28169 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:13:16.966784 28169 webserver.cc:460] Webserver started at http://127.19.58.254:41185/ using document root <none> and password file <none>
I20260808 02:13:16.967036 28169 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:16.967092 28169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:16.967221 28169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:16.968230 28169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "4e2775f670c3465e8a5cc9f3892342f6"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-1l3l"
server_key: "c432f6179c112775f849a8655a9d0344"
server_key_iv: "61a25c293f882e15551c449143cf7a02"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.968524 28169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "4e2775f670c3465e8a5cc9f3892342f6"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-1l3l"
server_key: "c432f6179c112775f849a8655a9d0344"
server_key_iv: "61a25c293f882e15551c449143cf7a02"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.969697 28169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:16.970294 28193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:16.970486 28169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:16.970570 28169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "4e2775f670c3465e8a5cc9f3892342f6"
format_stamp: "Formatted at 2026-08-08 02:13:16 on dist-test-slave-1l3l"
server_key: "c432f6179c112775f849a8655a9d0344"
server_key_iv: "61a25c293f882e15551c449143cf7a02"
server_key_version: "encryptionkey@0"
I20260808 02:13:16.970638 28169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:17.001411 28169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:17.003026 28169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:17.003147 28169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:17.006870 28169 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:41805
I20260808 02:13:17.006915 28272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:41805 every 8 connection(s)
I20260808 02:13:17.007217 28169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:17.008060 28273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:17.009573 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 28169
I20260808 02:13:17.009676 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:17.009871 19691 external_mini_cluster.cc:1428] Setting key ee18dc3db63b0d5fd263824f70b7296e
I20260808 02:13:17.010557 28273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6: Bootstrap starting.
I20260808 02:13:17.011093 28273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:17.011386 28273 log.cc:824] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:17.012130 28273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6: No bootstrap required, opened a new log
I20260808 02:13:17.014122 28273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2775f670c3465e8a5cc9f3892342f6" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41805 } }
I20260808 02:13:17.014250 28273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:17.014271 28273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e2775f670c3465e8a5cc9f3892342f6, State: Initialized, Role: FOLLOWER
I20260808 02:13:17.014377 28273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2775f670c3465e8a5cc9f3892342f6" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41805 } }
I20260808 02:13:17.014443 28273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:17.014463 28273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:17.014508 28273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:17.015050 28273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2775f670c3465e8a5cc9f3892342f6" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41805 } }
I20260808 02:13:17.015154 28273 leader_election.cc:302] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e2775f670c3465e8a5cc9f3892342f6; no voters:
I20260808 02:13:17.015333 28273 leader_election.cc:288] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:17.015368 28282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:17.015493 28282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e2775f670c3465e8a5cc9f3892342f6, State: Running, Role: LEADER
I20260808 02:13:17.015573 28282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2775f670c3465e8a5cc9f3892342f6" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41805 } }
I20260808 02:13:17.015597 28273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:17.015924 28282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e2775f670c3465e8a5cc9f3892342f6. Latest consensus state: current_term: 1 leader_uuid: "4e2775f670c3465e8a5cc9f3892342f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2775f670c3465e8a5cc9f3892342f6" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41805 } } }
I20260808 02:13:17.015995 28282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:17.016141 28285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e2775f670c3465e8a5cc9f3892342f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e2775f670c3465e8a5cc9f3892342f6" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41805 } } }
I20260808 02:13:17.016206 28285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:17.016324 28290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 02:13:17 dist-test-slave-1l3l krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:17.016964 28290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:17.018676 28290 catalog_manager.cc:1347] Generated new cluster ID: 6e90187598684e18826949be2f9d682e
I20260808 02:13:17.018729 28290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:17.020901 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.010375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34166 (local address 127.19.58.254:41805)
0808 02:13:17.010554 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.010561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.010585 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:13:17.011035 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.011041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.011086 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.011221 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.011995 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.012969 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.014041 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.014535 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.014633 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.017477 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.017480 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.017481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.017490 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.019162 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.019509 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.019512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.019513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.019621 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.019861 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.019862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.019862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.020198 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.020325 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.020590 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.020725 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260808 02:13:17.031520 28290 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:17.032249 28290 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:17.037196 28290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e2775f670c3465e8a5cc9f3892342f6: Generated new TSK 0
I20260808 02:13:17.037387 28290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:13:17.063874 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41805
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:17.176641 28314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:17.176855 28314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:17.176888 28314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:17.178025 28314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:17.178067 28314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:17.178100 28314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:17.179615 28314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:17.179881 28314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:17.180094 28314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:17.184839 28321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:17.184928 28314 server_base.cc:1034] running on GCE node
W20260808 02:13:17.184872 28325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:17.184861 28327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:17.185267 28314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:17.185483 28314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:17.186612 28314 hybrid_clock.cc:648] HybridClock initialized: now 1786155197186596 us; error 34 us; skew 500 ppm
Aug 08 02:13:17 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155197, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:17.189018 28314 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:13:17.189427 28314 webserver.cc:460] Webserver started at http://127.19.58.193:34595/ using document root <none> and password file <none>
I20260808 02:13:17.189620 28314 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:17.189659 28314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:17.189747 28314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:17.190698 28314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "620ae2cf6be04e4e9901c807c68883e0"
format_stamp: "Formatted at 2026-08-08 02:13:17 on dist-test-slave-1l3l"
server_key: "929c5f53a25f659ad6afe3048c21d913"
server_key_iv: "7c9eb544eb3fc51341ec5544978f381f"
server_key_version: "encryptionkey@0"
I20260808 02:13:17.190977 28314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "620ae2cf6be04e4e9901c807c68883e0"
format_stamp: "Formatted at 2026-08-08 02:13:17 on dist-test-slave-1l3l"
server_key: "929c5f53a25f659ad6afe3048c21d913"
server_key_iv: "7c9eb544eb3fc51341ec5544978f381f"
server_key_version: "encryptionkey@0"
I20260808 02:13:17.192233 28314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:13:17.192889 28337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:17.193040 28314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:17.193117 28314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "620ae2cf6be04e4e9901c807c68883e0"
format_stamp: "Formatted at 2026-08-08 02:13:17 on dist-test-slave-1l3l"
server_key: "929c5f53a25f659ad6afe3048c21d913"
server_key_iv: "7c9eb544eb3fc51341ec5544978f381f"
server_key_version: "encryptionkey@0"
I20260808 02:13:17.193178 28314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:17.205399 28314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:17.207146 28314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:17.207271 28314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:17.207490 28314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:17.207798 28314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:17.207834 28314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:17.207856 28314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:17.207880 28314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:17.213311 28314 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:43795
I20260808 02:13:17.213388 28498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:43795 every 8 connection(s)
I20260808 02:13:17.213665 28314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:17.219115 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 28314
I20260808 02:13:17.219211 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:17.219383 19691 external_mini_cluster.cc:1428] Setting key b8b6757988754fb0fc85c92ea60bf339
Aug 08 02:13:17 dist-test-slave-1l3l krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155197, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:17.222882 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.214475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:46673 (local address 127.19.58.254:41805)
0808 02:13:17.214550 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:17.214552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:17.214685 (+ 133us) server_negotiation.cc:407] Connection header received
0808 02:13:17.215634 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:17.215635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:17.215647 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:17.215696 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:17.216362 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.216944 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.217924 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.218122 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.218172 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.220258 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:17.220259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:17.220259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:17.220267 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:17.221616 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.221986 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.221987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.221987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.222041 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:17.222253 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:17.222254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:17.222255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:17.222363 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:17.222447 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:17.222699 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 02:13:17.222789 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 02:13:17.223037 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:17.214434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:41805 (local address 127.19.58.193:46673)
0808 02:13:17.214575 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:17.214611 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:17.215554 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:17.215785 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:17.215788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:17.216075 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:17.216300 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.216306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.217016 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.217017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.217865 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:17.217868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:17.218178 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:17.218179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:17.218267 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:17.218698 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:17.218706 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:17.220175 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:17.221692 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.221698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.221699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.221910 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.222096 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:17.222097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:17.222097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:17.222196 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:17.222447 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:17.222455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:17.222583 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 02:13:17.222670 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 02:13:17.222813 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260808 02:13:17.223568 28499 heartbeater.cc:347] Connected to a master server at 127.19.58.254:41805
I20260808 02:13:17.223664 28499 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:17.223863 28499 heartbeater.cc:511] Master 127.19.58.254:41805 requested a full tablet report, sending...
I20260808 02:13:17.224373 28219 ts_manager.cc:194] Registered new tserver with Master: 620ae2cf6be04e4e9901c807c68883e0 (127.19.58.193:43795)
I20260808 02:13:17.225009 28219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:46673
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
W20260808 02:13:18.155428 19691 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.19.58.194: real 0.935s user 0.000s sys 0.004s
I20260808 02:13:18.155866 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41805
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:13:18.225873 28499 heartbeater.cc:503] Master 127.19.58.254:41805 was elected leader, sending a full tablet report...
W20260808 02:13:18.235612 28508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:18.235837 28508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:18.235875 28508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:18.236986 28508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:18.237025 28508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:18.237059 28508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:18.238457 28508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:18.238690 28508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:18.238886 28508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:18.243492 28519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:18.243549 28508 server_base.cc:1034] running on GCE node
W20260808 02:13:18.243520 28518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:18.243486 28521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:18.243909 28508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:18.244105 28508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:18.245245 28508 hybrid_clock.cc:648] HybridClock initialized: now 1786155198245235 us; error 29 us; skew 500 ppm
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:18.252249 28508 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:13:18.252667 28508 webserver.cc:460] Webserver started at http://127.19.58.194:37773/ using document root <none> and password file <none>
I20260808 02:13:18.252874 28508 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:18.252920 28508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:18.253036 28508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:18.254001 28508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "a7fc04c9c96942238b0466f54d0dc6fc"
format_stamp: "Formatted at 2026-08-08 02:13:18 on dist-test-slave-1l3l"
server_key: "0638de9862097b260546bd78a908eba0"
server_key_iv: "eb5f4f41c881b105c7ab3ced3cc0e9e1"
server_key_version: "encryptionkey@0"
I20260808 02:13:18.254317 28508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "a7fc04c9c96942238b0466f54d0dc6fc"
format_stamp: "Formatted at 2026-08-08 02:13:18 on dist-test-slave-1l3l"
server_key: "0638de9862097b260546bd78a908eba0"
server_key_iv: "eb5f4f41c881b105c7ab3ced3cc0e9e1"
server_key_version: "encryptionkey@0"
I20260808 02:13:18.255548 28508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:18.256242 28534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:18.256385 28508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:18.256466 28508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "a7fc04c9c96942238b0466f54d0dc6fc"
format_stamp: "Formatted at 2026-08-08 02:13:18 on dist-test-slave-1l3l"
server_key: "0638de9862097b260546bd78a908eba0"
server_key_iv: "eb5f4f41c881b105c7ab3ced3cc0e9e1"
server_key_version: "encryptionkey@0"
I20260808 02:13:18.256528 28508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:18.264231 28508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:18.265803 28508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:18.265913 28508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:18.266110 28508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:18.266377 28508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:18.266418 28508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:18.266444 28508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:18.266462 28508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:18.271842 28508 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:40111
I20260808 02:13:18.271909 28704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:40111 every 8 connection(s)
I20260808 02:13:18.272159 28508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:18.280874 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:18.272806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:41927 (local address 127.19.58.254:41805)
0808 02:13:18.272935 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:18.272938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:18.273048 (+ 110us) server_negotiation.cc:407] Connection header received
0808 02:13:18.274012 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:18.274013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:18.274028 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:18.274084 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:18.274670 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.275187 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.276218 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.276435 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.276492 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:18.278409 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:18.278410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:18.278410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:18.278418 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:18.279570 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:18.279981 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:18.279982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:18.279983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:18.280042 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:18.280251 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:18.280252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:18.280252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:18.280367 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:18.280427 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:18.280674 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 02:13:18.280767 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260808 02:13:18.280979 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:18.272763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:41805 (local address 127.19.58.194:41927)
0808 02:13:18.272931 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:18.272974 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:18.273915 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:18.274158 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:18.274161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:18.274410 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:18.274582 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:18.274586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.275269 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.275270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:18.276134 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:18.276137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.276506 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.276507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:18.276618 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:18.277052 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:18.277060 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:18.278279 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:18.279644 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:18.279651 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:18.279653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:18.279913 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:18.280095 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:18.280096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:18.280096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:18.280196 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:18.280424 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:18.280431 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:18.280558 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 02:13:18.280642 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 02:13:18.280781 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260808 02:13:18.281076 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 28508
I20260808 02:13:18.281189 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:18.281371 19691 external_mini_cluster.cc:1428] Setting key 2c12f4b24823510c2f6c97528322c18a
I20260808 02:13:18.281896 28705 heartbeater.cc:347] Connected to a master server at 127.19.58.254:41805
I20260808 02:13:18.281983 28705 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:18.282163 28705 heartbeater.cc:511] Master 127.19.58.254:41805 requested a full tablet report, sending...
I20260808 02:13:18.282591 28218 ts_manager.cc:194] Registered new tserver with Master: a7fc04c9c96942238b0466f54d0dc6fc (127.19.58.194:40111)
I20260808 02:13:18.283023 28218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:41927
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:13:18.304324 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41805
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:18.383569 28715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:18.383785 28715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:18.383821 28715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:18.384886 28715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:18.384918 28715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:18.384941 28715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:18.386312 28715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:41805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:18.386489 28715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:18.386653 28715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:18.391286 28728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:18.391376 28725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:18.391286 28726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:18.391382 28715 server_base.cc:1034] running on GCE node
I20260808 02:13:18.391785 28715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:18.392005 28715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:18.393148 28715 hybrid_clock.cc:648] HybridClock initialized: now 1786155198393129 us; error 38 us; skew 500 ppm
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:18.395519 28715 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:13:18.396003 28715 webserver.cc:460] Webserver started at http://127.19.58.195:34403/ using document root <none> and password file <none>
I20260808 02:13:18.396224 28715 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:18.396271 28715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:18.396394 28715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:18.397320 28715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "5f99091099024f9d845485b272583bc7"
format_stamp: "Formatted at 2026-08-08 02:13:18 on dist-test-slave-1l3l"
server_key: "b9ab16d79e2e72c80b9a511573d07fdd"
server_key_iv: "f712089b0dd3685e200fc0cc7d0a1149"
server_key_version: "encryptionkey@0"
I20260808 02:13:18.397635 28715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "5f99091099024f9d845485b272583bc7"
format_stamp: "Formatted at 2026-08-08 02:13:18 on dist-test-slave-1l3l"
server_key: "b9ab16d79e2e72c80b9a511573d07fdd"
server_key_iv: "f712089b0dd3685e200fc0cc7d0a1149"
server_key_version: "encryptionkey@0"
I20260808 02:13:18.398819 28715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 02:13:18.399803 28739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:18.400005 28715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:18.400102 28715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "5f99091099024f9d845485b272583bc7"
format_stamp: "Formatted at 2026-08-08 02:13:18 on dist-test-slave-1l3l"
server_key: "b9ab16d79e2e72c80b9a511573d07fdd"
server_key_iv: "f712089b0dd3685e200fc0cc7d0a1149"
server_key_version: "encryptionkey@0"
I20260808 02:13:18.400172 28715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:18.408258 28715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:18.409766 28715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:18.409866 28715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:18.410027 28715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:18.410249 28715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:18.410276 28715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:18.410295 28715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:18.410315 28715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:18.415890 28715 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:35087
I20260808 02:13:18.415932 28915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:35087 every 8 connection(s)
I20260808 02:13:18.416234 28715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:18.418455 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 28715
I20260808 02:13:18.418561 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:18.418751 19691 external_mini_cluster.cc:1428] Setting key 93813cfdb40458e221b07b3f59fa55f7
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:18.424742 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:18.416908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:56449 (local address 127.19.58.254:41805)
0808 02:13:18.416972 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:18.416974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:18.417141 (+ 167us) server_negotiation.cc:407] Connection header received
0808 02:13:18.417985 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:18.417987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:18.417997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:18.418059 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:18.418748 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.419366 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.420400 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.420601 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.420653 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:18.422514 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:18.422515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:18.422515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:18.422521 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:18.423802 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:18.424106 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:18.424107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:18.424107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:18.424154 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:18.424304 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:18.424305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:18.424305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:18.424394 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:18.424436 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:18.424618 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 02:13:18.424680 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:13:18.424866 28918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:18.416870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:41805 (local address 127.19.58.195:56449)
0808 02:13:18.417023 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:18.417079 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:18.417878 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:18.418108 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:18.418110 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:18.418386 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:18.418644 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:18.418650 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.419443 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.419444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:18.420332 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:18.420335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.420659 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.420660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:18.420749 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:18.421129 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:18.421136 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:18.422432 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:18.423858 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:18.423863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:18.423864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:18.424052 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:18.424190 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:18.424191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:18.424191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:18.424266 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:18.424434 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:18.424436 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:18.424531 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 02:13:18.424602 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:13:18.424705 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260808 02:13:18.425267 28916 heartbeater.cc:347] Connected to a master server at 127.19.58.254:41805
I20260808 02:13:18.425351 28916 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:18.425506 28916 heartbeater.cc:511] Master 127.19.58.254:41805 requested a full tablet report, sending...
I20260808 02:13:18.425832 28218 ts_manager.cc:194] Registered new tserver with Master: 5f99091099024f9d845485b272583bc7 (127.19.58.195:35087)
I20260808 02:13:18.426172 28218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:56449
I20260808 02:13:18.430251 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/test-admin.keytab.
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:18.445729 19691 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 02:13:18.447183 28926 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:18.446525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.19.58.193:43795)
0808 02:13:18.446685 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:18.446690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:18.446700 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:13:18.446725 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:18.446726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:18.446740 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:18.446798 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:18.447122 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33720: BlockingRecv error: recv got EOF from 127.0.0.1:33720 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
W20260808 02:13:18.447371 19691 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5cc)
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:18.448313 19691 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:18.449116 19691 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:13:18.455811 28926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:18.449583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.19.58.193:43795)
0808 02:13:18.449630 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:18.449631 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:18.449642 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:13:18.449697 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:18.449698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:18.449712 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:18.449754 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:18.450098 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.450789 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.451438 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.451750 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.451801 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:18.453630 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:18.453638 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:18.453638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:18.453645 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:18.454946 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:18.455222 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:18.455225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:18.455225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:18.455273 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:18.455403 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:18.455404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:18.455404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:18.455512 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:18.455618 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:18.455654 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 02:13:18.455745 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 02:13:18.457480 28926 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:18.456918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33738 (local address 127.19.58.193:43795)
0808 02:13:18.456958 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:18.456960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:18.457004 (+ 44us) server_negotiation.cc:407] Connection header received
0808 02:13:18.457046 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:18.457047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:18.457055 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:18.457101 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:18.457421 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33738: BlockingRecv error: recv got EOF from 127.0.0.1:33738 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 02:13:18.457544 19691 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:18.458453 19691 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:18.459316 19691 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:13:18.465911 28926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:18.459934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33746 (local address 127.19.58.193:43795)
0808 02:13:18.459972 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:18.459974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:18.460028 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:13:18.460117 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:18.460119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:18.460128 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:18.460166 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:18.460483 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.460973 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.461632 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.461822 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.461868 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:18.463668 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:18.463669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:18.463669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:18.463675 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:18.464958 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:18.465260 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:18.465260 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:18.465260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:18.465303 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:18.465506 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:18.465507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:18.465507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:18.465610 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:18.465690 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:18.465765 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 02:13:18.465830 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 02:13:18.467145 28926 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:18.466730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33754 (local address 127.19.58.193:43795)
0808 02:13:18.466789 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:18.466790 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:18.466800 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:13:18.466857 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:18.466858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:18.466865 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:18.466898 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:18.467103 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33754: BlockingRecv error: recv got EOF from 127.0.0.1:33754 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 02:13:18.467222 19691 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786155170542577-19691-0/krb5kdc/krb5cc' of type FILE
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:18.468199 19691 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:18.469069 19691 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:13:18.475035 28926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:18.469511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33762 (local address 127.19.58.193:43795)
0808 02:13:18.469546 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:18.469548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:18.469612 (+ 64us) server_negotiation.cc:407] Connection header received
0808 02:13:18.469648 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:18.469649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:18.469655 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:18.469696 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:18.469999 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.470419 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.471014 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:18.471181 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:18.471234 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:18.473026 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:18.473027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:18.473028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:18.473034 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:18.474207 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:18.474503 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:18.474504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:18.474504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:18.474555 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:18.474691 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:18.474691 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:18.474691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:18.474766 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:18.474827 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:18.474907 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 02:13:18.474960 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:13:18.475589 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 28314
I20260808 02:13:18.492936 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 28508
I20260808 02:13:18.505648 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 28715
I20260808 02:13:18.513211 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 28169
2026-08-08T02:13:18Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2857 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28947](info): set up 2 sockets
Aug 08 02:13:18 dist-test-slave-1l3l krb5kdc[28947](info): commencing operation
krb5kdc: starting...
W20260808 02:13:19.628681 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:19 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.254.keytab.
Entry for principal oryx/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.254.keytab.
I20260808 02:13:19.740293 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:32909
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.254.keytab
--principal=oryx/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:32909
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:19.819550 28966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:19.819770 28966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:19.819797 28966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:19.820904 28966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:19.820948 28966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:19.820962 28966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:19.820974 28966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:19.820984 28966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:19.822382 28966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:32909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.254.keytab
--principal=oryx/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:32909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:19.822597 28966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:19.822794 28966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:19.827360 28976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:19.827477 28981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:19.827374 28977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:19.827608 28966 server_base.cc:1034] running on GCE node
I20260808 02:13:19.827839 28966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:19.828080 28966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:19.829205 28966 hybrid_clock.cc:648] HybridClock initialized: now 1786155199829194 us; error 27 us; skew 500 ppm
Aug 08 02:13:19 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155199, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:19.831599 28966 init.cc:377] Logged in from keytab as oryx/127.19.58.254@KRBTEST.COM (short username oryx)
I20260808 02:13:19.832042 28966 webserver.cc:460] Webserver started at http://127.19.58.254:39019/ using document root <none> and password file <none>
I20260808 02:13:19.832226 28966 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:19.832269 28966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:19.832384 28966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:19.833379 28966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "b3aec6d749834e9c99e150a0e4ba7eb7"
format_stamp: "Formatted at 2026-08-08 02:13:19 on dist-test-slave-1l3l"
server_key: "caa88524f66c63ba21f3d0c376604b8d"
server_key_iv: "0fcdeed972c5aa16694486f5ed98bfab"
server_key_version: "encryptionkey@0"
I20260808 02:13:19.833701 28966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "b3aec6d749834e9c99e150a0e4ba7eb7"
format_stamp: "Formatted at 2026-08-08 02:13:19 on dist-test-slave-1l3l"
server_key: "caa88524f66c63ba21f3d0c376604b8d"
server_key_iv: "0fcdeed972c5aa16694486f5ed98bfab"
server_key_version: "encryptionkey@0"
I20260808 02:13:19.834947 28966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 02:13:19.835603 28990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:19.835783 28966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:19.835865 28966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "b3aec6d749834e9c99e150a0e4ba7eb7"
format_stamp: "Formatted at 2026-08-08 02:13:19 on dist-test-slave-1l3l"
server_key: "caa88524f66c63ba21f3d0c376604b8d"
server_key_iv: "0fcdeed972c5aa16694486f5ed98bfab"
server_key_version: "encryptionkey@0"
I20260808 02:13:19.835938 28966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:19.848585 28966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:19.850595 28966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:19.850736 28966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:19.854703 28966 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:32909
I20260808 02:13:19.854751 29076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:32909 every 8 connection(s)
I20260808 02:13:19.855078 28966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:19.856169 29079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:19.858445 29079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7: Bootstrap starting.
I20260808 02:13:19.858995 29079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:19.859227 29079 log.cc:824] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:19.859917 29079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7: No bootstrap required, opened a new log
I20260808 02:13:19.861585 29079 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } }
I20260808 02:13:19.861720 29079 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:19.861742 29079 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3aec6d749834e9c99e150a0e4ba7eb7, State: Initialized, Role: FOLLOWER
I20260808 02:13:19.861850 29079 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } }
I20260808 02:13:19.861923 29079 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:19.861945 29079 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:19.861971 29079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:19.862722 29079 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } }
I20260808 02:13:19.862807 29079 leader_election.cc:302] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3aec6d749834e9c99e150a0e4ba7eb7; no voters:
I20260808 02:13:19.862926 29079 leader_election.cc:288] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:19.862983 29082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:19.863152 29079 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:19.863190 29082 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 1 LEADER]: Becoming Leader. State: Replica: b3aec6d749834e9c99e150a0e4ba7eb7, State: Running, Role: LEADER
I20260808 02:13:19.863371 29082 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } }
I20260808 02:13:19.863735 29083 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } }
I20260808 02:13:19.863742 29086 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3aec6d749834e9c99e150a0e4ba7eb7. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } }
I20260808 02:13:19.863832 29083 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:19.863837 29086 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:19.864053 29095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:19.864507 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 28966
I20260808 02:13:19.864598 29095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:19.864620 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:19.864828 19691 external_mini_cluster.cc:1428] Setting key e082af0edc4649900bd9fae95c4a61a7
I20260808 02:13:19.866357 29095 catalog_manager.cc:1347] Generated new cluster ID: 1ccb96e17b9044c9b63ad047597abf1b
I20260808 02:13:19.866410 29095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:13:19 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:19.876029 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:19.865325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58968 (local address 127.19.58.254:32909)
0808 02:13:19.865507 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:19.865516 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:19.865690 (+ 174us) server_negotiation.cc:407] Connection header received
0808 02:13:19.865823 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:19.865831 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:19.865904 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:19.866041 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:19.866746 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.868171 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.869176 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:19.869766 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:19.869890 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:19.872557 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:19.872561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:19.872561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:19.872573 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:19.874432 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:19.874810 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:19.874812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:19.874813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:19.874910 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:19.875094 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:19.875095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:19.875095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:19.875383 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:19.875522 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:19.875624 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 02:13:19.875782 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260808 02:13:19.878535 29095 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:19.878990 29095 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:19.881621 29095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7: Generated new TSK 0
I20260808 02:13:19.881760 29095 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab.
Entry for principal oryx/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab.
I20260808 02:13:19.913812 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:32909
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab
--principal=oryx/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:19.992941 29117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:19.993129 29117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:19.993151 29117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:19.994216 29117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:19.994246 29117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:19.994270 29117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:19.995626 29117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab
--principal=oryx/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:32909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:19.995839 29117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:19.996021 29117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:20.000794 29125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.000805 29126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.000833 29117 server_base.cc:1034] running on GCE node
W20260808 02:13:20.000900 29128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.001204 29117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:20.001403 29117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:20.002535 29117 hybrid_clock.cc:648] HybridClock initialized: now 1786155200002517 us; error 36 us; skew 500 ppm
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:20.004870 29117 init.cc:377] Logged in from keytab as oryx/127.19.58.193@KRBTEST.COM (short username oryx)
I20260808 02:13:20.005265 29117 webserver.cc:460] Webserver started at http://127.19.58.193:44247/ using document root <none> and password file <none>
I20260808 02:13:20.005472 29117 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:20.005522 29117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:20.005642 29117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:20.006625 29117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "bceb22edd6e34183a043a4744086e1d7"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "2e8f365a7c8ed7a4a01246d62296c619"
server_key_iv: "34b437697356cedd569c7fb36db878bd"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.006948 29117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "bceb22edd6e34183a043a4744086e1d7"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "2e8f365a7c8ed7a4a01246d62296c619"
server_key_iv: "34b437697356cedd569c7fb36db878bd"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.008213 29117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 02:13:20.008951 29137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.009105 29117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 02:13:20.009177 29117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "bceb22edd6e34183a043a4744086e1d7"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "2e8f365a7c8ed7a4a01246d62296c619"
server_key_iv: "34b437697356cedd569c7fb36db878bd"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.009235 29117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:20.036269 29117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:20.038127 29117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:20.038293 29117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:20.038522 29117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:20.038807 29117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:20.038841 29117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.038870 29117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:20.038903 29117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.044412 29117 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:35443
I20260808 02:13:20.044482 29313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:35443 every 8 connection(s)
I20260808 02:13:20.044755 29117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:20.048609 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 29117
I20260808 02:13:20.048703 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:20.048908 19691 external_mini_cluster.cc:1428] Setting key 04a51c7056a4fd8e8a386cfc08bcec33
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:20.054046 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.045455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:59063 (local address 127.19.58.254:32909)
0808 02:13:20.045501 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.045503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.045696 (+ 193us) server_negotiation.cc:407] Connection header received
0808 02:13:20.046201 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.046203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.046219 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.046263 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.047195 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.047688 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.048823 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.049081 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.049156 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.051408 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.051409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.051410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.051421 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.052737 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.053130 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.053131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.053132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.053184 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.053440 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.053441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.053442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.053547 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.053605 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.053858 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.053954 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:13:20.054221 29319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.045407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.193:59063)
0808 02:13:20.045580 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.045615 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.046101 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.046346 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.046351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.046903 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.047091 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.047095 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.047784 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.047785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.048740 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.048744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.049172 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.049174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.049313 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.049783 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.049794 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.051328 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.052828 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.052836 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.052838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.053062 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.053239 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.053240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.053241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.053372 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.053605 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.053613 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.053741 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.053828 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.054007 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260808 02:13:20.054680 29315 heartbeater.cc:347] Connected to a master server at 127.19.58.254:32909
I20260808 02:13:20.054783 29315 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:20.054976 29315 heartbeater.cc:511] Master 127.19.58.254:32909 requested a full tablet report, sending...
I20260808 02:13:20.055480 29012 ts_manager.cc:194] Registered new tserver with Master: bceb22edd6e34183a043a4744086e1d7 (127.19.58.193:35443)
I20260808 02:13:20.056131 29012 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:59063
WARNING: no policy specified for oryx/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab.
Entry for principal oryx/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab.
I20260808 02:13:20.074483 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:32909
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab
--principal=oryx/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:20.154006 29322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:20.154209 29322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:20.154232 29322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:20.155349 29322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:20.155388 29322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:20.155413 29322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:20.156834 29322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab
--principal=oryx/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:32909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:20.157040 29322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:20.157228 29322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:20.162160 29337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.162252 29334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.162184 29335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.162256 29322 server_base.cc:1034] running on GCE node
I20260808 02:13:20.162577 29322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:20.162808 29322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:20.163944 29322 hybrid_clock.cc:648] HybridClock initialized: now 1786155200163926 us; error 29 us; skew 500 ppm
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:20.166350 29322 init.cc:377] Logged in from keytab as oryx/127.19.58.194@KRBTEST.COM (short username oryx)
I20260808 02:13:20.166750 29322 webserver.cc:460] Webserver started at http://127.19.58.194:40319/ using document root <none> and password file <none>
I20260808 02:13:20.166942 29322 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:20.166990 29322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:20.167112 29322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:20.168145 29322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "1d3360d0edc04a44b0260e36dcf59dfd"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "f94a4f360bf7b1e253658f967a716a50"
server_key_iv: "55fe5bfaceb7601170efc006c5f2f449"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.168483 29322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "1d3360d0edc04a44b0260e36dcf59dfd"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "f94a4f360bf7b1e253658f967a716a50"
server_key_iv: "55fe5bfaceb7601170efc006c5f2f449"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.169654 29322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:20.170289 29345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.170423 29322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:20.170495 29322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "1d3360d0edc04a44b0260e36dcf59dfd"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "f94a4f360bf7b1e253658f967a716a50"
server_key_iv: "55fe5bfaceb7601170efc006c5f2f449"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.170558 29322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:20.183599 29322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:20.185279 29322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:20.185415 29322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:20.185629 29322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:20.185912 29322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:20.185945 29322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.185978 29322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:20.186003 29322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.191212 29322 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:42639
I20260808 02:13:20.191283 29510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:42639 every 8 connection(s)
I20260808 02:13:20.191545 29322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:20.198820 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 29322
I20260808 02:13:20.198907 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:20.199123 19691 external_mini_cluster.cc:1428] Setting key d360651c21dd9bc8794fa5bc505b407a
I20260808 02:13:20.199954 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.192207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:34107 (local address 127.19.58.254:32909)
0808 02:13:20.192251 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.192253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.192445 (+ 192us) server_negotiation.cc:407] Connection header received
0808 02:13:20.192903 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.192904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.192914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.192967 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.193805 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.194321 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.195226 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.195420 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.195468 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.197386 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.197388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.197388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.197395 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.198475 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.198817 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.198818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.198819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.198873 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.199190 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.199192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.199192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.199316 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.199443 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.199773 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.199860 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 02:13:20.200085 29514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.192174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.194:34107)
0808 02:13:20.192329 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.192368 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.192814 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.193030 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.193033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.193565 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.193741 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.193745 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.194396 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.194396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.195165 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.195168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.195474 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.195475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.195564 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.195978 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.195986 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.197293 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.198549 (+ 1256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.198555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.198557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.198752 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.198970 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.198971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.198972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.199114 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.199445 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.199453 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.199608 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.199697 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.199876 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260808 02:13:20.200528 29511 heartbeater.cc:347] Connected to a master server at 127.19.58.254:32909
I20260808 02:13:20.200616 29511 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:20.200783 29511 heartbeater.cc:511] Master 127.19.58.254:32909 requested a full tablet report, sending...
I20260808 02:13:20.201116 29012 ts_manager.cc:194] Registered new tserver with Master: 1d3360d0edc04a44b0260e36dcf59dfd (127.19.58.194:42639)
I20260808 02:13:20.201465 29012 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:34107
WARNING: no policy specified for oryx/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab.
Entry for principal oryx/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab.
I20260808 02:13:20.224045 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:32909
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab
--principal=oryx/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:20.303254 29517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:20.303452 29517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:20.303473 29517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:20.304852 29517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:20.304903 29517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:20.304937 29517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:20.306326 29517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab
--principal=oryx/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:32909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:20.306522 29517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:20.306696 29517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:20.311457 29523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.311486 29527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.311452 29525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.311646 29517 server_base.cc:1034] running on GCE node
I20260808 02:13:20.311913 29517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:20.312191 29517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:20.313374 29517 hybrid_clock.cc:648] HybridClock initialized: now 1786155200313353 us; error 41 us; skew 500 ppm
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:20.315775 29517 init.cc:377] Logged in from keytab as oryx/127.19.58.195@KRBTEST.COM (short username oryx)
I20260808 02:13:20.316210 29517 webserver.cc:460] Webserver started at http://127.19.58.195:38725/ using document root <none> and password file <none>
I20260808 02:13:20.316409 29517 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:20.316458 29517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:20.316586 29517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:20.317618 29517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "1dc6ea81f1ba4e1083af64eddea9f858"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "ee54b4a6363ea1a32ecd63fe7a8bc863"
server_key_iv: "c6f6f2b9724246268de82a3e981d58c6"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.317955 29517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "1dc6ea81f1ba4e1083af64eddea9f858"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "ee54b4a6363ea1a32ecd63fe7a8bc863"
server_key_iv: "c6f6f2b9724246268de82a3e981d58c6"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.319284 29517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:20.319996 29536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.320147 29517 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.320225 29517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "1dc6ea81f1ba4e1083af64eddea9f858"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "ee54b4a6363ea1a32ecd63fe7a8bc863"
server_key_iv: "c6f6f2b9724246268de82a3e981d58c6"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.320283 29517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:20.331509 29517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:20.333104 29517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:20.333231 29517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:20.333425 29517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:20.333699 29517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:20.333732 29517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.333758 29517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:20.333784 29517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.339447 29517 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:37345
I20260808 02:13:20.339506 29709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:37345 every 8 connection(s)
I20260808 02:13:20.339833 29517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:20.347636 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.340416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:33285 (local address 127.19.58.254:32909)
0808 02:13:20.340488 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.340490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.340655 (+ 165us) server_negotiation.cc:407] Connection header received
0808 02:13:20.341126 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.341126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.341137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.341184 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.342014 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.342505 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.343395 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.343572 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.343617 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.345588 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.345589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.345590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.345595 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.346647 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.346973 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.346974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.346974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.347014 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.347189 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.347190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.347190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.347279 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.347322 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.347517 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.347572 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 02:13:20.347805 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.340391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.195:33285)
0808 02:13:20.340539 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.340571 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.341048 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.341236 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.341239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.341762 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.341948 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.341953 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.342566 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.342566 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.343332 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.343334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.343626 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.343627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.343744 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.344216 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.344223 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.345502 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.346709 (+ 1207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.346715 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.346716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.346918 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.347057 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.347057 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.347058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.347149 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.347319 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.347322 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.347422 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.347496 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.347605 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260808 02:13:20.348260 29711 heartbeater.cc:347] Connected to a master server at 127.19.58.254:32909
I20260808 02:13:20.348345 29711 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:20.348506 29711 heartbeater.cc:511] Master 127.19.58.254:32909 requested a full tablet report, sending...
I20260808 02:13:20.348548 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 29517
I20260808 02:13:20.348650 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:20.348837 19691 external_mini_cluster.cc:1428] Setting key c47e9e8c1c148b8904e749d450a1e249
I20260808 02:13:20.348879 29012 ts_manager.cc:194] Registered new tserver with Master: 1dc6ea81f1ba4e1083af64eddea9f858 (127.19.58.195:37345)
I20260808 02:13:20.349321 29012 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:33285
I20260808 02:13:20.349715 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.58.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.253.keytab.
Entry for principal oryx/127.19.58.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.253.keytab.
I20260808 02:13:20.375468 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.253:33461
--webserver_interface=127.19.58.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.253.keytab
--principal=oryx/127.19.58.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:32909,127.19.58.253:33461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:20.456781 29719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:20.456981 29719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:20.457002 29719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:20.458071 29719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:20.458104 29719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:20.458117 29719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:20.458127 29719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:20.458138 29719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:20.459517 29719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:32909,127.19.58.253:33461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.253.keytab
--principal=oryx/127.19.58.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.253:33461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.58.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
W20260808 02:13:20.459612 29719 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.19.58.254:32909,127.19.58.253:33461'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 02:13:20.459739 29719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:20.459942 29719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:20.464493 29729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.464494 29728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.464613 29719 server_base.cc:1034] running on GCE node
W20260808 02:13:20.464568 29734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.464905 29719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:20.465161 29719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:20.466364 29719 hybrid_clock.cc:648] HybridClock initialized: now 1786155200466351 us; error 35 us; skew 500 ppm
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:20.468760 29719 init.cc:377] Logged in from keytab as oryx/127.19.58.253@KRBTEST.COM (short username oryx)
I20260808 02:13:20.469143 29719 webserver.cc:460] Webserver started at http://127.19.58.253:39357/ using document root <none> and password file <none>
I20260808 02:13:20.469340 29719 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:20.469422 29719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:20.469544 29719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:20.470537 29719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data/instance:
uuid: "520b53d52d04411aadfc1dbe16ecd079"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "04cec4365abfb14fcbbfed121fbce523"
server_key_iv: "c30b14082da502c2b54f237afdc93baf"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.470856 29719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal/instance:
uuid: "520b53d52d04411aadfc1dbe16ecd079"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "04cec4365abfb14fcbbfed121fbce523"
server_key_iv: "c30b14082da502c2b54f237afdc93baf"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.472087 29719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:20.472813 29747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.472972 29719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:20.473050 29719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal
uuid: "520b53d52d04411aadfc1dbe16ecd079"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "04cec4365abfb14fcbbfed121fbce523"
server_key_iv: "c30b14082da502c2b54f237afdc93baf"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.473107 29719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:20.491097 29719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:20.492766 29719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:20.492874 29719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:20.496507 29719 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.253:33461
I20260808 02:13:20.496555 29818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.253:33461 every 8 connection(s)
I20260808 02:13:20.496840 29719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
I20260808 02:13:20.497771 29819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:20.499220 29819 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:13:20.500891 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 29719
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.253@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.58.253@KRBTEST.COM
I20260808 02:13:20.507784 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.499873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.19.58.254:32909)
0808 02:13:20.499931 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.499933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.500137 (+ 204us) server_negotiation.cc:407] Connection header received
0808 02:13:20.500614 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.500615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.500627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.500680 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.501328 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.501905 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.502833 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.503025 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.503080 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.504995 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.504996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.504996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.505003 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.506377 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.506747 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.506748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.506749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.506808 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.507072 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.507073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.507073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.507268 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.507354 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.507617 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.507678 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 02:13:20.508028 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.499854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:58970)
0808 02:13:20.500029 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.500062 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.500516 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.500767 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.500771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.501058 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.501236 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.501241 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.502006 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.502007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.502754 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.502756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.503093 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.503094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.503202 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.503615 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.503623 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.504917 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.506440 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.506447 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.506448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.506672 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.506870 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.506871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.506871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.507011 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.507330 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.507337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.507485 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.507598 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.507795 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260808 02:13:20.508563 29819 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:13:20.509486 29824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.501420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.19.58.253:33461)
0808 02:13:20.501571 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.501576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.501602 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:13:20.501622 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.501624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.501642 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.501703 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.502233 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.502945 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.503613 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.503935 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.503994 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.506317 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.506324 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.506324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.506330 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.508076 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.508446 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.508449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.508449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.508509 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.508956 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.508957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.508957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.509086 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.509193 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.509318 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.509392 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.253@KRBTEST.COM for oryx/127.19.58.253@KRBTEST.COM
I20260808 02:13:20.514851 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.508772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:33461 (local address 127.0.0.1:56652)
0808 02:13:20.508814 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.508823 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.508873 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.509044 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.509045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.509181 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.509227 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.509229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.509999 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.510000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.510510 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.510513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.510822 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.510823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.510872 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.511189 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.511193 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.512288 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.514001 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.514002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.514002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.514143 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.514296 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.514297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.514297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.514347 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.514562 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.514563 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.514596 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.514670 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.514756 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 02:13:20.514950 29825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.508831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56652 (local address 127.19.58.253:33461)
0808 02:13:20.508955 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.508957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.508962 (+ 5us) server_negotiation.cc:407] Connection header received
0808 02:13:20.508976 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.508976 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.508983 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.509036 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.509297 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.509920 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.510577 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.510769 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.510819 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.512356 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.512357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.512358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.512363 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.513932 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.514199 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.514200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.514200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.514247 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.514418 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.514419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.514419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.514509 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.514561 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.514696 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.514806 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260808 02:13:20.516052 29819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: Bootstrap starting.
I20260808 02:13:20.516597 29819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:20.516804 29819 log.cc:824] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:20.517442 29819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: No bootstrap required, opened a new log
I20260808 02:13:20.518702 29819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } }
I20260808 02:13:20.518806 29819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:20.518831 29819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520b53d52d04411aadfc1dbe16ecd079, State: Initialized, Role: FOLLOWER
I20260808 02:13:20.518918 29819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } }
I20260808 02:13:20.519143 29819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:20.519117 29828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } } }
I20260808 02:13:20.519191 29828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:13:20.522301 29843 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:20.522358 29843 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:20.523926 29719 master_runner.cc:428] Detected that this master 520b53d52d04411aadfc1dbe16ecd079 is joining an existing cluster
I20260808 02:13:20.524032 29719 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.58.253:33461
I20260808 02:13:20.530989 29825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.525240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56662 (local address 127.19.58.253:33461)
0808 02:13:20.525279 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.525280 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.525337 (+ 57us) server_negotiation.cc:407] Connection header received
0808 02:13:20.525421 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.525421 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.525431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.525474 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.525734 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.526203 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.527187 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.527504 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.527575 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.528458 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.528460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.528460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.528466 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.529928 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.530264 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.530265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.530265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.530309 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.530483 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.530485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.530485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.530600 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.530665 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.530830 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.530906 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:13:20.531236 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.525107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:33461 (local address 127.0.0.1:56662)
0808 02:13:20.525287 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.525295 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.525357 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.525479 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.525480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.525631 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.525671 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.525672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.526266 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.526267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.527104 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.527106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.527588 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.527589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.527632 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.528030 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.528035 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.528352 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.530010 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.530010 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.530011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.530197 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.530381 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.530382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.530382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.530435 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.530666 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.530667 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.530708 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.531064 (+ 356us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.531143 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":59,"threads_started":1}
I20260808 02:13:20.531810 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.524940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.19.58.254:32909)
0808 02:13:20.524985 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.524986 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.525070 (+ 84us) server_negotiation.cc:407] Connection header received
0808 02:13:20.525118 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.525119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.525129 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.525189 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.525540 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.526283 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.527626 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.527939 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.528011 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.529155 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.529156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.529156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.529164 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.530363 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.530727 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.530728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.530728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.530780 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.530947 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.530948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.530948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.531057 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.531119 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.531613 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.531687 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:13:20.532083 29853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.524861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:58978)
0808 02:13:20.525003 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.525013 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.525071 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.525192 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.525193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.525380 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.525447 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.525449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.526360 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.526361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.527118 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.527119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.528056 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.528057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.528120 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.528702 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.528706 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.529076 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.530443 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.530444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.530444 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.530641 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.530834 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.530835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.530835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.530897 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.531180 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.531181 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.531221 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.531880 (+ 659us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.531981 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155199, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.254@KRBTEST.COM for oryx/127.19.58.253@KRBTEST.COM
I20260808 02:13:20.540299 29825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.534017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56670 (local address 127.19.58.253:33461)
0808 02:13:20.534058 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.534060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.534182 (+ 122us) server_negotiation.cc:407] Connection header received
0808 02:13:20.534253 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.534254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.534264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.534307 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.534669 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.535103 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.535853 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.536037 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.536086 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.537975 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.537976 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.537976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.537982 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.539218 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.539587 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.539588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.539588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.539637 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.539823 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.539824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.539825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.539922 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.539975 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.540149 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.540217 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:13:20.540323 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.533976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:33461 (local address 127.0.0.1:56670)
0808 02:13:20.534096 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.534128 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.534193 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.534312 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.534315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.534556 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.534598 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.534599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.535169 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.535170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.535778 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.535780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.536092 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.536093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.536166 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.536568 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.536572 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.537900 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.539300 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.539324 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.539326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.539522 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.539689 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.539690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.539690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.539772 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.539976 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.539984 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.540034 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.540105 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.540222 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":70,"threads_started":1}
I20260808 02:13:20.540810 29011 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.58.253:33461: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079"
member_type: NON_VOTER
last_known_addr {
host: "127.19.58.253"
port: 33461
}
attrs {
promote: true
}
}
dest_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7"
cas_config_opid_index: -1
I20260808 02:13:20.541096 29011 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } }
I20260808 02:13:20.541522 29859 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 520b53d52d04411aadfc1dbe16ecd079 (127.19.58.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } } }
I20260808 02:13:20.541814 29786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:20.542584 29859 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.58.253:33461
I20260808 02:13:20.542691 29786 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 FOLLOWER]: Refusing update from remote peer b3aec6d749834e9c99e150a0e4ba7eb7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 02:13:20.542740 29859 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } } }
I20260808 02:13:20.542800 29859 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:20.542888 29719 master.cc:537] Master@127.19.58.253:33461 shutting down...
I20260808 02:13:20.542995 29859 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 02:13:20.543659 28992 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 02:13:20.544477 28992 consensus_peers.cc:487] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 -> Peer 520b53d52d04411aadfc1dbe16ecd079 (127.19.58.253:33461): Couldn't send request to peer 520b53d52d04411aadfc1dbe16ecd079. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 02:13:20.545817 29719 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:20.545897 29719 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:20.545945 29719 tablet_replica.cc:330] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: stopping tablet replica
I20260808 02:13:20.558427 29719 master.cc:552] Master@127.19.58.253:33461 shutdown complete.
I20260808 02:13:20.558466 29719 master_runner.cc:305] Clearing existing system tablet
I20260808 02:13:20.558580 29719 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:20.558619 29719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:20.559168 29719 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:20.559600 29866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.559744 29719 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:20.559794 29719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.559814 29719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal
uuid: "520b53d52d04411aadfc1dbe16ecd079"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "04cec4365abfb14fcbbfed121fbce523"
server_key_iv: "c30b14082da502c2b54f237afdc93baf"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.560571 29719 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:20.561821 29719 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 02:13:20.561879 29719 log.cc:1196] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 02:13:20.562055 29719 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: Deleting consensus metadata
I20260808 02:13:20.562160 29719 master_runner.cc:315] Copying system tablet from 127.19.58.254:32909
I20260808 02:13:20.562748 29719 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: tablet copy: Beginning tablet copy session from remote peer at address 127.19.58.254:32909
I20260808 02:13:20.568900 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.563113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.19.58.254:32909)
0808 02:13:20.563157 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.563159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.563223 (+ 64us) server_negotiation.cc:407] Connection header received
0808 02:13:20.563296 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.563297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.563307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.563364 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.563739 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.564305 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.565328 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.565521 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.565567 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.566512 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.566513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.566514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.566520 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.567804 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.568241 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.568243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.568243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.568306 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.568496 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.568497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.568497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.568594 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.568642 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.568776 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.568836 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 02:13:20.569161 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.563045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:58992)
0808 02:13:20.563163 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.563174 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.563239 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.563368 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.563368 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.563584 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.563651 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.563653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.564373 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.564374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.565253 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.565255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.565580 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.565581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.565646 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.566041 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.566046 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.566436 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.567869 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.567870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.567870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.568086 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.568376 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.568377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.568377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.568439 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.568644 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.568645 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.568689 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.568759 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.568856 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
I20260808 02:13:20.569517 29044 tablet_copy_service.cc:140] P b3aec6d749834e9c99e150a0e4ba7eb7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 520b53d52d04411aadfc1dbe16ecd079 ({username='oryx', principal='oryx/127.19.58.253@KRBTEST.COM'} at 127.0.0.1:58992)
I20260808 02:13:20.569602 29044 tablet_copy_service.cc:161] P b3aec6d749834e9c99e150a0e4ba7eb7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 520b53d52d04411aadfc1dbe16ecd079 at {username='oryx', principal='oryx/127.19.58.253@KRBTEST.COM'} at 127.0.0.1:58992: session id = 520b53d52d04411aadfc1dbe16ecd079-00000000000000000000000000000000
I20260808 02:13:20.570206 29044 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:13:20.570731 29719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 02:13:20.571568 29719 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: tablet copy: Starting download of 0 data blocks...
I20260808 02:13:20.571762 29719 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: tablet copy: Starting download of 1 WAL segments...
I20260808 02:13:20.572536 29719 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:13:20.573263 29044 tablet_copy_service.cc:342] P b3aec6d749834e9c99e150a0e4ba7eb7: Request end of tablet copy session 520b53d52d04411aadfc1dbe16ecd079-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.58.253@KRBTEST.COM'} at 127.0.0.1:58992
I20260808 02:13:20.573313 29044 tablet_copy_service.cc:433] P b3aec6d749834e9c99e150a0e4ba7eb7: ending tablet copy session 520b53d52d04411aadfc1dbe16ecd079-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 520b53d52d04411aadfc1dbe16ecd079
I20260808 02:13:20.575752 29719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:20.576804 29876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.576939 29878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.576969 29875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.579881 29719 server_base.cc:1034] running on GCE node
I20260808 02:13:20.579977 29719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:20.580108 29719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:20.581230 29719 hybrid_clock.cc:648] HybridClock initialized: now 1786155200581222 us; error 24 us; skew 500 ppm
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:20.582428 29719 init.cc:377] Logged in from keytab as oryx/127.19.58.253@KRBTEST.COM (short username oryx)
I20260808 02:13:20.582687 29719 webserver.cc:460] Webserver started at http://127.19.58.253:37913/ using document root <none> and password file <none>
I20260808 02:13:20.582809 29719 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:20.582844 29719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:20.583379 29719 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:20.583876 29887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.583994 29719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.584053 29719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal
uuid: "520b53d52d04411aadfc1dbe16ecd079"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "04cec4365abfb14fcbbfed121fbce523"
server_key_iv: "c30b14082da502c2b54f237afdc93baf"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.584105 29719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:20.613227 29719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:20.614929 29719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:20.618132 29719 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.253:33461
I20260808 02:13:20.618162 29961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.253:33461 every 8 connection(s)
I20260808 02:13:20.618502 29719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
I20260808 02:13:20.619151 29962 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:13:20.619580 29962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: Bootstrap starting.
I20260808 02:13:20.621769 29962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:13:20.621948 29962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: Bootstrap complete.
I20260808 02:13:20.622107 29962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } }
I20260808 02:13:20.622176 29962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 520b53d52d04411aadfc1dbe16ecd079, State: Initialized, Role: LEARNER
I20260808 02:13:20.622211 29962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } }
I20260808 02:13:20.622381 29962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:20.622376 29965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } } }
I20260808 02:13:20.622466 29965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:13:20.623682 29982 catalog_manager.cc:1259] Loaded cluster ID: 1ccb96e17b9044c9b63ad047597abf1b
I20260808 02:13:20.623746 29982 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: loading cluster ID for follower catalog manager: success
I20260808 02:13:20.624701 29982 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: acquiring CA information for follower catalog manager: success
I20260808 02:13:20.624971 29982 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:13:20.667296 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.661460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56704 (local address 127.19.58.253:33461)
0808 02:13:20.661583 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.661585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.661592 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:20.661615 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.661616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.661625 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.661687 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.662212 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.662882 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.663634 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.663891 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.663950 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.665154 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.665155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.665155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.665162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.666319 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.666614 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.666615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.666615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.666670 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.666830 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.666830 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.666831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.666938 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.667020 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.667118 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.667197 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for oryx/127.19.58.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.252.keytab.
Entry for principal oryx/127.19.58.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.252.keytab.
I20260808 02:13:20.692625 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.252:34863
--webserver_interface=127.19.58.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.252.keytab
--principal=oryx/127.19.58.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:32909,127.19.58.253:33461,127.19.58.252:34863
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:20.772573 29988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:20.772766 29988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:20.772796 29988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:20.773900 29988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:20.773945 29988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:20.773964 29988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:20.773980 29988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:20.774004 29988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:20.775472 29988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:32909,127.19.58.253:33461,127.19.58.252:34863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.252.keytab
--principal=oryx/127.19.58.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.252:34863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.58.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:20.775729 29988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:20.775941 29988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:20.780395 29994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.780439 29997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.780611 29988 server_base.cc:1034] running on GCE node
W20260808 02:13:20.780718 29995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.780925 29988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:20.781175 29988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:20.782310 29988 hybrid_clock.cc:648] HybridClock initialized: now 1786155200782282 us; error 44 us; skew 500 ppm
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:20.784703 29988 init.cc:377] Logged in from keytab as oryx/127.19.58.252@KRBTEST.COM (short username oryx)
I20260808 02:13:20.785145 29988 webserver.cc:460] Webserver started at http://127.19.58.252:38539/ using document root <none> and password file <none>
I20260808 02:13:20.785362 29988 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:20.785429 29988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:20.785555 29988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:20.786535 29988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data/instance:
uuid: "3ed51ae392984dc281859d6a42d27856"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "84b75d39906b11659d7100f2010e88c9"
server_key_iv: "332658cc74d142ab264913900479d470"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.786861 29988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal/instance:
uuid: "3ed51ae392984dc281859d6a42d27856"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "84b75d39906b11659d7100f2010e88c9"
server_key_iv: "332658cc74d142ab264913900479d470"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.788117 29988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:13:20.788790 30009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.788928 29988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.789012 29988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal
uuid: "3ed51ae392984dc281859d6a42d27856"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "84b75d39906b11659d7100f2010e88c9"
server_key_iv: "332658cc74d142ab264913900479d470"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.789077 29988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:20.795994 29988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:20.797895 29988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:20.798023 29988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:20.801775 29988 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.252:34863
I20260808 02:13:20.801805 30089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.252:34863 every 8 connection(s)
I20260808 02:13:20.802093 29988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
I20260808 02:13:20.802970 30090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:20.804333 30090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:13:20.807353 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 29988
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.252@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.58.252@KRBTEST.COM
I20260808 02:13:20.812819 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.805013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58994 (local address 127.19.58.254:32909)
0808 02:13:20.805090 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.805092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.805158 (+ 66us) server_negotiation.cc:407] Connection header received
0808 02:13:20.805576 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.805577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.805587 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.805635 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.806153 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.806634 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.807648 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.807901 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.807995 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.810147 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.810148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.810148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.810155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.811505 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.811909 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.811910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.811911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.811972 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.812227 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.812228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.812228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.812336 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.812392 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.812691 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.812755 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 02:13:20.813048 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.804876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:58994)
0808 02:13:20.805058 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.805091 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.805503 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.805697 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.805699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.805920 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.806094 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.806097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.806703 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.806704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.807572 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.807575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.808032 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.808033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.808164 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.808583 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.808593 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.810062 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.811577 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.811584 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.811586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.811845 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.812033 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.812034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.812034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.812160 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.812403 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.812409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.812549 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.812657 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.812806 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":79,"threads_started":1}
I20260808 02:13:20.813542 30090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:13:20.815825 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.807951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.19.58.252:34863)
0808 02:13:20.808098 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.808102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.808118 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:20.808138 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.808140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.808158 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.808214 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.808880 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.809540 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.810404 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.810692 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.810751 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.813289 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.813293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.813293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.813300 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.814754 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.815069 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.815072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.815073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.815126 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.815290 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.815291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.815291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.815436 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.815567 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.815638 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.815735 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.252@KRBTEST.COM for oryx/127.19.58.253@KRBTEST.COM
I20260808 02:13:20.819131 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.813817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:33461 (local address 127.0.0.1:56718)
0808 02:13:20.813859 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.813868 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.813901 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.814052 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.814052 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.814169 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.814212 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.814213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.814801 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.814802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.815270 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.815271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.815613 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.815614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.815676 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.816254 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.816259 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.817278 (+ 1019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.818448 (+ 1170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.818449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.818449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.818593 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.818723 (+ 130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.818724 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.818724 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.818763 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.818942 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.818943 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.818974 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.819017 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.819078 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 02:13:20.819150 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.813897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56718 (local address 127.19.58.253:33461)
0808 02:13:20.813958 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.813959 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.813967 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:20.813988 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.813989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.814002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.814045 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.814277 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.814736 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.815347 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.815558 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.815610 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.817354 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.817355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.817355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.817362 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.818382 (+ 1020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.818645 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.818646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.818646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.818689 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.818793 (+ 104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.818794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.818794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.818886 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.818928 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.819034 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.819093 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:13:20.819516 30090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.252@KRBTEST.COM for oryx/127.19.58.252@KRBTEST.COM
I20260808 02:13:20.825037 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.819731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34863 (local address 127.0.0.1:36294)
0808 02:13:20.819812 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.819822 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.819867 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.819973 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.819973 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.820088 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.820112 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.820113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.820674 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.820675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.821119 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.821120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.821441 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.821442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.821484 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.821762 (+ 278us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.821766 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.822880 (+ 1114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.824290 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.824291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.824291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.824454 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.824591 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.824592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.824592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.824634 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.824849 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.824849 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.824878 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.824917 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.824985 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 02:13:20.825110 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.819772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36294 (local address 127.19.58.252:34863)
0808 02:13:20.819815 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.819817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.819854 (+ 37us) server_negotiation.cc:407] Connection header received
0808 02:13:20.819913 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.819914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.819923 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.819969 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.820168 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.820610 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.821181 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.821386 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.821443 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.822957 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.822958 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.822959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.822964 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.824225 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.824509 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.824509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.824509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.824550 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.824689 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.824690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.824690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.824795 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.824846 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.824939 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.825029 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":28}
I20260808 02:13:20.826145 30090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Bootstrap starting.
I20260808 02:13:20.826860 30090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:20.827116 30090 log.cc:824] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:20.827767 30090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: No bootstrap required, opened a new log
I20260808 02:13:20.829077 30090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } }
I20260808 02:13:20.829183 30090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:20.829214 30090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed51ae392984dc281859d6a42d27856, State: Initialized, Role: FOLLOWER
I20260808 02:13:20.829325 30090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } }
I20260808 02:13:20.829581 30090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:20.829562 30097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } } }
I20260808 02:13:20.829686 30097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:13:20.831151 30114 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:20.831199 30114 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:20.833288 29988 master_runner.cc:428] Detected that this master 3ed51ae392984dc281859d6a42d27856 is joining an existing cluster
I20260808 02:13:20.833330 29988 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.58.252:34863
I20260808 02:13:20.840701 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.834227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:59004)
0808 02:13:20.834383 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.834392 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.834436 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.834744 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.834745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.834905 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.834966 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.834968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.835869 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.835870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.836737 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.836739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.837214 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.837215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.837271 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.837757 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.837760 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.838052 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.839691 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.839692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.839692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.839961 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.840157 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.840157 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.840157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.840204 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.840468 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.840469 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.840514 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.840564 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.840634 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260808 02:13:20.840762 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.834262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59004 (local address 127.19.58.254:32909)
0808 02:13:20.834584 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.834586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.834594 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:20.834615 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.834615 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.834625 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.834670 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.835039 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.835789 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.836817 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.837109 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.837159 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.838137 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.838139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.838139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.838146 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.839621 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.840040 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.840041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.840041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.840104 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.840244 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.840245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.840245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.840408 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.840473 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.840601 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.840676 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297}
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155199, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.254@KRBTEST.COM for oryx/127.19.58.252@KRBTEST.COM
I20260808 02:13:20.858853 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.842476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34863 (local address 127.0.0.1:36308)
0808 02:13:20.842533 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.842544 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.842587 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.842766 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.842767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.842919 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.842957 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.842958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.843553 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.843554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.844413 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.844416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.844778 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.844778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.844847 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.845159 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.845164 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.846430 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.857864 (+ 11434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.857865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.857865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.858035 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.858242 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.858243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.858244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.858302 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.858583 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.858585 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.858629 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.858673 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.858767 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 02:13:20.859392 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.834431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.19.58.253:33461)
0808 02:13:20.834468 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.834469 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.834637 (+ 168us) server_negotiation.cc:407] Connection header received
0808 02:13:20.834678 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.834679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.834689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.834744 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.835128 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.835913 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.836933 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.837210 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.837250 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.856686 (+ 19436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.856688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.856688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.856695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.858159 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.858609 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.858610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.858610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.858663 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.858858 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.858859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.858859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.858979 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.859020 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.859207 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.859287 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:13:20.859699 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.842526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.19.58.252:34863)
0808 02:13:20.842657 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.842659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.842666 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:20.842695 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.842696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.842703 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.842754 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.843020 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.843492 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.844511 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.844717 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.844768 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.846545 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.846546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.846546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.846552 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.857790 (+ 11238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.858094 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.858095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.858096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.858161 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.858373 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.858374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.858374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.858502 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.859171 (+ 669us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.859225 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.859317 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260808 02:13:20.859740 30126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.834239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:33461 (local address 127.0.0.1:56734)
0808 02:13:20.834572 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.834582 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.834631 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.834789 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.834790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.834973 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.835035 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.835037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.836007 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.836008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.836836 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.836838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.837322 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.837323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.837429 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.856146 (+ 18717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.856151 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.856582 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.858247 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.858248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.858248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.858525 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.858726 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.858727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.858727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.858793 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.859065 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.859065 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.859104 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.859508 (+ 404us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.859614 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":78,"threads_started":1}
I20260808 02:13:20.860678 29011 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.58.252:34863: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3ed51ae392984dc281859d6a42d27856"
member_type: NON_VOTER
last_known_addr {
host: "127.19.58.252"
port: 34863
}
attrs {
promote: true
}
}
dest_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7"
cas_config_opid_index: 5
I20260808 02:13:20.860924 29011 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } }
I20260808 02:13:20.861246 29860 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3ed51ae392984dc281859d6a42d27856 (127.19.58.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.861394 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.834736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.19.58.252:34863)
0808 02:13:20.834776 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.834778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.835109 (+ 331us) server_negotiation.cc:407] Connection header received
0808 02:13:20.835177 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.835177 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.835183 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.835204 (+ 21us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.835570 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.836220 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.837136 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.837369 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.837417 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.858637 (+ 21220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.858639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.858639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.858644 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.860251 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.860610 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.860611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.860611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.860667 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.860892 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.860893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.860894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.861011 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.861069 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.861213 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.861300 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:13:20.861738 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.834466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34863 (local address 127.0.0.1:36304)
0808 02:13:20.835065 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.835073 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.835126 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.835243 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.835244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.835441 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.835500 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.835502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.836294 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.836295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.837062 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.837064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.837477 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.837478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.837545 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.858112 (+ 20567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.858117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.858551 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.860341 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.860341 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.860342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.860525 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.860730 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.860731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.860732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.860806 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.861090 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.861090 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.861122 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.861555 (+ 433us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.861641 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":71,"threads_started":1}
I20260808 02:13:20.862217 29860 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.58.252:34863
I20260808 02:13:20.862301 29860 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.862370 29860 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:20.862411 29988 master.cc:537] Master@127.19.58.252:34863 shutting down...
W20260808 02:13:20.862871 28996 consensus_peers.cc:487] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 -> Peer 3ed51ae392984dc281859d6a42d27856 (127.19.58.252:34863): Couldn't send request to peer 3ed51ae392984dc281859d6a42d27856. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 02:13:20.865653 29988 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:20.865739 29988 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:20.865779 29988 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: stopping tablet replica
I20260808 02:13:20.865918 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.862654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56750 (local address 127.19.58.253:33461)
0808 02:13:20.862704 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.862706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.862734 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:13:20.862791 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.862792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.862815 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.862875 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:20.863055 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.863744 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.865065 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.865627 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.865692 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.865757 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.865824 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.865854 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 02:13:20.865916 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.862573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:33461 (local address 127.0.0.1:56750)
0808 02:13:20.862692 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.862702 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.862746 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.862946 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.862948 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.862949 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:20.862986 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.862987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.863836 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.863837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.864920 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.864924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.865692 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.865693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.865752 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.865772 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.865819 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.865853 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260808 02:13:20.866175 29932 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 LEARNER]: Refusing update from remote peer b3aec6d749834e9c99e150a0e4ba7eb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 02:13:20.866523 30129 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 02:13:20.867048 29932 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3ed51ae392984dc281859d6a42d27856 (127.19.58.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.868242 30131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.868316 30131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:13:20.868219 29965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3aec6d749834e9c99e150a0e4ba7eb7. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: NON_VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: true } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.868500 29965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:13:20.868984 30129 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7: attempting to promote NON_VOTER 520b53d52d04411aadfc1dbe16ecd079 to VOTER
I20260808 02:13:20.869149 30129 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } }
I20260808 02:13:20.869540 29932 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 LEARNER]: Refusing update from remote peer b3aec6d749834e9c99e150a0e4ba7eb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 02:13:20.869590 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.869489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34863 (local address 127.0.0.1:36310)
0808 02:13:20.869522 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.869568 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 02:13:20.869774 29859 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 02:13:20.869853 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.869780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34863 (local address 127.0.0.1:36316)
0808 02:13:20.869808 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.869834 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260808 02:13:20.869916 28996 consensus_peers.cc:487] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 -> Peer 3ed51ae392984dc281859d6a42d27856 (127.19.58.252:34863): Couldn't send request to peer 3ed51ae392984dc281859d6a42d27856. Status: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34863: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:13:20.870323 30129 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 520b53d52d04411aadfc1dbe16ecd079 (127.19.58.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.870484 29932 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 520b53d52d04411aadfc1dbe16ecd079 (127.19.58.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.871210 29965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.871287 29965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:20.871456 29860 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.871538 29860 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:20.871462 30129 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.871595 30129 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:20.879276 29988 master.cc:552] Master@127.19.58.252:34863 shutdown complete.
I20260808 02:13:20.879315 29988 master_runner.cc:305] Clearing existing system tablet
I20260808 02:13:20.879436 29988 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:20.879477 29988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:20.880045 29988 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:20.880482 30136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.880613 29988 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:20.880678 29988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.880703 29988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal
uuid: "3ed51ae392984dc281859d6a42d27856"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "84b75d39906b11659d7100f2010e88c9"
server_key_iv: "332658cc74d142ab264913900479d470"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.881482 29988 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:20.882771 29988 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 02:13:20.882841 29988 log.cc:1196] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 02:13:20.883038 29988 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Deleting consensus metadata
I20260808 02:13:20.883176 29988 master_runner.cc:315] Copying system tablet from 127.19.58.254:32909
I20260808 02:13:20.883844 29988 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: tablet copy: Beginning tablet copy session from remote peer at address 127.19.58.254:32909
I20260808 02:13:20.889555 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.884136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59010 (local address 127.19.58.254:32909)
0808 02:13:20.884183 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.884186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.884254 (+ 68us) server_negotiation.cc:407] Connection header received
0808 02:13:20.884323 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.884323 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.884333 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.884377 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.884711 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.885216 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.885915 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.886115 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.886167 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.887362 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.887363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.887363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.887370 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.888603 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.888891 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.888892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.888892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.888944 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.889102 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.889102 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.889102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.889204 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.889258 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.889390 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.889464 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:13:20.889555 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.884069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:59010)
0808 02:13:20.884206 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:20.884214 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:20.884268 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:20.884392 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:20.884392 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:20.884588 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:20.884646 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.884648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.885287 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.885288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.885843 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:20.885844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.886172 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.886173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:20.886236 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.887019 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:20.887023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:20.887284 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:20.888673 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.888674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.888675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.888832 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.888997 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:20.888998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:20.888998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:20.889053 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:20.889259 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:20.889260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:20.889303 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:13:20.889362 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:13:20.889460 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260808 02:13:20.889750 29044 tablet_copy_service.cc:140] P b3aec6d749834e9c99e150a0e4ba7eb7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3ed51ae392984dc281859d6a42d27856 ({username='oryx', principal='oryx/127.19.58.252@KRBTEST.COM'} at 127.0.0.1:59010)
I20260808 02:13:20.889806 29044 tablet_copy_service.cc:161] P b3aec6d749834e9c99e150a0e4ba7eb7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3ed51ae392984dc281859d6a42d27856 at {username='oryx', principal='oryx/127.19.58.252@KRBTEST.COM'} at 127.0.0.1:59010: session id = 3ed51ae392984dc281859d6a42d27856-00000000000000000000000000000000
I20260808 02:13:20.890199 29044 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:13:20.890617 29988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 02:13:20.891662 29988 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: tablet copy: Starting download of 0 data blocks...
I20260808 02:13:20.891844 29988 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: tablet copy: Starting download of 1 WAL segments...
I20260808 02:13:20.892535 29988 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:13:20.893347 29044 tablet_copy_service.cc:342] P b3aec6d749834e9c99e150a0e4ba7eb7: Request end of tablet copy session 3ed51ae392984dc281859d6a42d27856-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.58.252@KRBTEST.COM'} at 127.0.0.1:59010
I20260808 02:13:20.893400 29044 tablet_copy_service.cc:433] P b3aec6d749834e9c99e150a0e4ba7eb7: ending tablet copy session 3ed51ae392984dc281859d6a42d27856-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3ed51ae392984dc281859d6a42d27856
I20260808 02:13:20.895511 29988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:20.896605 30145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.896656 30150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:20.896775 30146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:20.899852 29988 server_base.cc:1034] running on GCE node
I20260808 02:13:20.899945 29988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:20.900074 29988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:20.901182 29988 hybrid_clock.cc:648] HybridClock initialized: now 1786155200901177 us; error 23 us; skew 500 ppm
Aug 08 02:13:20 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155200, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:20.902354 29988 init.cc:377] Logged in from keytab as oryx/127.19.58.252@KRBTEST.COM (short username oryx)
I20260808 02:13:20.902599 29988 webserver.cc:460] Webserver started at http://127.19.58.252:37965/ using document root <none> and password file <none>
I20260808 02:13:20.902709 29988 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:20.902743 29988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:20.903270 29988 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:20.903755 30160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.903870 29988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:20.903930 29988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal
uuid: "3ed51ae392984dc281859d6a42d27856"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "84b75d39906b11659d7100f2010e88c9"
server_key_iv: "332658cc74d142ab264913900479d470"
server_key_version: "encryptionkey@0"
I20260808 02:13:20.903983 29988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:20.922235 29988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:20.923853 29988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:20.926992 29988 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.252:34863
I20260808 02:13:20.927165 30236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.252:34863 every 8 connection(s)
I20260808 02:13:20.927387 29988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
I20260808 02:13:20.928114 30239 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:13:20.928510 30239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Bootstrap starting.
I20260808 02:13:20.930697 30239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:13:20.930892 30239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Bootstrap complete.
I20260808 02:13:20.931083 30239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } }
I20260808 02:13:20.931152 30239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3ed51ae392984dc281859d6a42d27856, State: Initialized, Role: LEARNER
I20260808 02:13:20.931186 30239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } }
I20260808 02:13:20.931353 30239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:20.931362 30244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:20.931471 30244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: This master's current role is: LEARNER
I20260808 02:13:20.932766 30263 catalog_manager.cc:1259] Loaded cluster ID: 1ccb96e17b9044c9b63ad047597abf1b
I20260808 02:13:20.932807 30263 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: loading cluster ID for follower catalog manager: success
I20260808 02:13:20.933729 30263 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: acquiring CA information for follower catalog manager: success
I20260808 02:13:20.933991 30263 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:13:20.973875 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:20.968205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36324 (local address 127.19.58.252:34863)
0808 02:13:20.968325 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:20.968327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:20.968334 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:20.968354 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:20.968355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:20.968365 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:20.968443 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:20.968992 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.969632 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.970295 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:20.970491 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:20.970545 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:20.971659 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:20.971660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:20.971661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:20.971667 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:20.972862 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.973162 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.973163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.973163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.973241 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:20.973413 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:20.973414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:20.973414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:20.973520 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:20.973605 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:20.973700 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 02:13:20.973779 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260808 02:13:20.974380 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 29117
I20260808 02:13:20.988435 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:35443
--local_ip_for_outbound_sockets=127.19.58.193
--tserver_master_addrs=127.19.58.254:32909
--webserver_port=44247
--webserver_interface=127.19.58.193
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab
--principal=oryx/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.58.254:32909,127.19.58.253:33461
--tserver_master_addrs=127.19.58.254:32909,127.19.58.253:33461,127.19.58.252:34863 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:21.071256 30266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:21.071446 30266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:21.071467 30266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:21.072664 30266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:21.072710 30266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:21.072746 30266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:21.074203 30266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab
--principal=oryx/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:35443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=44247
--tserver_master_addrs=127.19.58.254:32909,127.19.58.253:33461,127.19.58.252:34863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:21.074393 30266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:21.074568 30266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:21.079229 30273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:21.079247 30277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:21.079357 30275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:21.079355 30266 server_base.cc:1034] running on GCE node
I20260808 02:13:21.079661 30266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:21.079888 30266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:21.081027 30266 hybrid_clock.cc:648] HybridClock initialized: now 1786155201081009 us; error 30 us; skew 500 ppm
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:21.083365 30266 init.cc:377] Logged in from keytab as oryx/127.19.58.193@KRBTEST.COM (short username oryx)
I20260808 02:13:21.083811 30266 webserver.cc:460] Webserver started at http://127.19.58.193:44247/ using document root <none> and password file <none>
I20260808 02:13:21.084041 30266 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:21.084105 30266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:21.085341 30266 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:21.085846 30287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.085997 30266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.086076 30266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "bceb22edd6e34183a043a4744086e1d7"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "2e8f365a7c8ed7a4a01246d62296c619"
server_key_iv: "34b437697356cedd569c7fb36db878bd"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.086357 30266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:21.106148 30266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:21.107841 30266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:21.107968 30266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:21.108160 30266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:21.108436 30266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:21.108486 30266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.108523 30266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:21.108547 30266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.114229 30266 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:35443
I20260808 02:13:21.114292 30461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:35443 every 8 connection(s)
I20260808 02:13:21.114596 30266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.252@KRBTEST.COM
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.253@KRBTEST.COM
I20260808 02:13:21.123479 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 30266
I20260808 02:13:21.123651 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 29322
I20260808 02:13:21.124799 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.115271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:37411 (local address 127.19.58.252:34863)
0808 02:13:21.115334 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.115336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.115597 (+ 261us) server_negotiation.cc:407] Connection header received
0808 02:13:21.116171 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.116173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.116187 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.116215 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.117150 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.117906 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.118914 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.119179 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.119277 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.121528 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.121529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.121529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.121536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.123181 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.123683 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.123684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.123685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.123776 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.124063 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.124064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.124064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.124201 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.124242 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.124615 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.124706 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:13:21.125319 30473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.115217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34863 (local address 127.19.58.193:37411)
0808 02:13:21.115531 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.115539 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.116054 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.116292 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.116296 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.116797 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.117033 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.117039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.117983 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.117984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.118839 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.118841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.119281 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.119282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.119347 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.119988 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.119992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.121445 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.123288 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.123296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.123298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.123584 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.123852 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.123853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.123853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.123980 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.124289 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.124296 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.124463 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.124924 (+ 461us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.125097 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":53,"spinlock_wait_cycles":7552,"thread_start_us":113,"threads_started":1}
I20260808 02:13:21.125342 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.115284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:45829 (local address 127.19.58.253:33461)
0808 02:13:21.115347 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.115349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.115569 (+ 220us) server_negotiation.cc:407] Connection header received
0808 02:13:21.116168 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.116170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.116183 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.116214 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.117140 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.117626 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.118858 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.119109 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.119151 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.122726 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.122727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.122727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.122734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.124228 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.124571 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.124572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.124572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.124629 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.124814 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.124815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.124815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.124959 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.125002 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.125193 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.125264 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 02:13:21.125743 30471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.115216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:33461 (local address 127.19.58.193:45829)
0808 02:13:21.115445 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.115487 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.116050 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.116291 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.116296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.116797 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.117033 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.117039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.117723 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.117724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.118758 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.118761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.119226 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.119227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.119302 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.119851 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.119855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.122651 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.124304 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.124305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.124305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.124486 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.124698 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.124699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.124699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.124753 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.125048 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.125048 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.125085 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.125543 (+ 458us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.125636 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":22,"thread_start_us":102,"threads_started":1}
I20260808 02:13:21.130831 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.115500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.193:50059)
0808 02:13:21.115733 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.115742 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.116058 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.116535 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.116536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.116800 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.117032 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.117037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.117997 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.117998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.118675 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.118677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.119053 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.119053 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.119184 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.119800 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.119810 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.122412 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.129771 (+ 7359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.129780 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.129780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.129969 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.130175 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.130177 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.130177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.130265 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.130524 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.130525 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.130576 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.130644 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.130730 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":42,"thread_start_us":58,"threads_started":1}
I20260808 02:13:21.130925 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.115659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:50059 (local address 127.19.58.254:32909)
0808 02:13:21.115698 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.115717 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.115784 (+ 67us) server_negotiation.cc:407] Connection header received
0808 02:13:21.116135 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.116136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.116149 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.116199 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.117141 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.117904 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.118759 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.118977 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.119039 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.122508 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.122509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.122510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.122518 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.129691 (+ 7173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.130032 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.130033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.130033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.130094 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.130339 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.130340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.130340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.130459 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.130509 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.130692 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.130792 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:13:21.131448 30465 heartbeater.cc:347] Connected to a master server at 127.19.58.254:32909
I20260808 02:13:21.131552 30465 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.131815 30465 heartbeater.cc:511] Master 127.19.58.254:32909 requested a full tablet report, sending...
I20260808 02:13:21.132310 29011 ts_manager.cc:194] Re-registered known tserver with Master: bceb22edd6e34183a043a4744086e1d7 (127.19.58.193:35443)
I20260808 02:13:21.133015 29011 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:50059
I20260808 02:13:21.136783 30462 heartbeater.cc:347] Connected to a master server at 127.19.58.252:34863
I20260808 02:13:21.136843 30462 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.136921 30462 heartbeater.cc:511] Master 127.19.58.252:34863 requested a full tablet report, sending...
I20260808 02:13:21.137790 30184 ts_manager.cc:194] Registered new tserver with Master: bceb22edd6e34183a043a4744086e1d7 (127.19.58.193:35443)
I20260808 02:13:21.140000 30467 heartbeater.cc:347] Connected to a master server at 127.19.58.253:33461
I20260808 02:13:21.140064 30467 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.140111 30467 heartbeater.cc:511] Master 127.19.58.253:33461 requested a full tablet report, sending...
I20260808 02:13:21.140468 29915 ts_manager.cc:194] Registered new tserver with Master: bceb22edd6e34183a043a4744086e1d7 (127.19.58.193:35443)
I20260808 02:13:21.141052 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:42639
--local_ip_for_outbound_sockets=127.19.58.194
--tserver_master_addrs=127.19.58.254:32909
--webserver_port=40319
--webserver_interface=127.19.58.194
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab
--principal=oryx/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.58.254:32909,127.19.58.253:33461
--tserver_master_addrs=127.19.58.254:32909,127.19.58.253:33461,127.19.58.252:34863 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:21.220749 30475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:21.220945 30475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:21.220968 30475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:21.222101 30475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:21.222183 30475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:21.222208 30475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:21.223783 30475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab
--principal=oryx/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:42639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=40319
--tserver_master_addrs=127.19.58.254:32909,127.19.58.253:33461,127.19.58.252:34863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:21.224012 30475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:21.224237 30475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:21.228849 30484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:21.228863 30487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:21.228878 30485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:21.229074 30475 server_base.cc:1034] running on GCE node
I20260808 02:13:21.229252 30475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:21.229537 30475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:21.230695 30475 hybrid_clock.cc:648] HybridClock initialized: now 1786155201230676 us; error 30 us; skew 500 ppm
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:21.233247 30475 init.cc:377] Logged in from keytab as oryx/127.19.58.194@KRBTEST.COM (short username oryx)
I20260808 02:13:21.233696 30475 webserver.cc:460] Webserver started at http://127.19.58.194:40319/ using document root <none> and password file <none>
I20260808 02:13:21.233911 30475 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:21.233973 30475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:21.235271 30475 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:21.235968 30495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.236140 30475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:21.236228 30475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "1d3360d0edc04a44b0260e36dcf59dfd"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "f94a4f360bf7b1e253658f967a716a50"
server_key_iv: "55fe5bfaceb7601170efc006c5f2f449"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.236534 30475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:21.246483 30475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:21.248409 30475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:21.248510 30475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:21.248668 30475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:21.248890 30475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:21.248919 30475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.248937 30475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:21.248955 30475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.254294 30475 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:42639
I20260808 02:13:21.254330 30655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:42639 every 8 connection(s)
I20260808 02:13:21.254624 30475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:21.255959 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 30475
I20260808 02:13:21.256110 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 29517
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.253@KRBTEST.COM
I20260808 02:13:21.264006 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.254981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34863 (local address 127.0.0.1:36328)
0808 02:13:21.255091 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.255104 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.255159 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.255341 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.255342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.255344 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:21.255391 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.255393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.261565 (+ 6172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.261567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.262384 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.262388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.263570 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.263572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.263660 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.263793 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.263857 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.263918 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 02:13:21.264242 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.255098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.19.58.252:34863)
0808 02:13:21.255148 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.255150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.255161 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:13:21.255230 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.255231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.255256 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.255335 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:21.255443 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.256195 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.262482 (+ 6287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.263265 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.263370 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.263484 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.264119 (+ 635us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.264170 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:13:21.265343 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.255747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:45937 (local address 127.19.58.253:33461)
0808 02:13:21.255790 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.255791 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.255967 (+ 176us) server_negotiation.cc:407] Connection header received
0808 02:13:21.256361 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.256363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.256374 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.256399 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.257439 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.258013 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.258985 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.259243 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.259296 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.262358 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.262359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.262360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.262368 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.263815 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.264522 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.264524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.264524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.264578 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.264787 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.264787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.264788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.264889 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.264923 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.265192 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.265259 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:13:21.266140 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.255546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:33461 (local address 127.19.58.194:45937)
0808 02:13:21.255908 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.255917 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.256276 (+ 359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.256516 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.256520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.257122 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.257345 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.257350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.258097 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.258098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.258903 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.258905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.259319 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.259321 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.259514 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.260051 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.260061 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.262260 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.263892 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.263898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.263900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.264446 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.264635 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.264636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.264636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.264736 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.264957 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.264961 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.265087 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.265656 (+ 569us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.265878 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":67,"threads_started":1}
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.252@KRBTEST.COM
I20260808 02:13:21.269311 30660 heartbeater.cc:347] Connected to a master server at 127.19.58.253:33461
I20260808 02:13:21.269441 30660 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.269686 30660 heartbeater.cc:511] Master 127.19.58.253:33461 requested a full tablet report, sending...
I20260808 02:13:21.270067 30200 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 02:13:21.270177 29915 ts_manager.cc:194] Registered new tserver with Master: 1d3360d0edc04a44b0260e36dcf59dfd (127.19.58.194:42639)
I20260808 02:13:21.270282 30244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3aec6d749834e9c99e150a0e4ba7eb7. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:21.270380 30244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: This master's current role is: LEARNER
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:21.274302 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.256191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:43459 (local address 127.19.58.252:34863)
0808 02:13:21.264339 (+ 8148us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.264341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.264350 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:21.264369 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.264370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.264381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.264455 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.264966 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.265640 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.266686 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.266904 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.266960 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.271013 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.271014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.271015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.271023 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.272761 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.273298 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.273300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.273300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.273373 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.273640 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.273641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.273642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.273792 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.273845 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.274105 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.274197 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8132,"thread_start_us":66,"threads_started":1}
I20260808 02:13:21.274681 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.255644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34863 (local address 127.19.58.194:43459)
0808 02:13:21.256019 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.256026 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.258975 (+ 2949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.264544 (+ 5569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.264546 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.264772 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.264852 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.264855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.265740 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.265740 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.266599 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.266602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.266985 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.266987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.267061 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.267454 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.267460 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.270892 (+ 3432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.272864 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.272865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.272866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.273189 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.273472 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.273473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.273473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.273548 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.273899 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.273900 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.273953 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.274439 (+ 486us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.274554 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":86,"threads_started":1}
I20260808 02:13:21.275048 30657 heartbeater.cc:347] Connected to a master server at 127.19.58.252:34863
I20260808 02:13:21.275095 30657 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.275163 30657 heartbeater.cc:511] Master 127.19.58.252:34863 requested a full tablet report, sending...
I20260808 02:13:21.275502 30184 ts_manager.cc:194] Registered new tserver with Master: 1d3360d0edc04a44b0260e36dcf59dfd (127.19.58.194:42639)
I20260808 02:13:21.276968 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.269762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:34053 (local address 127.19.58.254:32909)
0808 02:13:21.269823 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.269825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.269833 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:21.269853 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.269854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.269864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.269910 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.270352 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.270947 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.272209 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.272432 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.272461 (+ 29us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.274182 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.274183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.274183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.274190 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.275911 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.276229 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.276231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.276231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.276280 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.276447 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.276448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.276448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.276618 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.276701 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.276828 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.276896 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:13:21.277029 30662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.255365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.194:34053)
0808 02:13:21.255538 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.255581 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.256181 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.269953 (+ 13772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.269954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.270103 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.270209 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.270211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.271111 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.271112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.272131 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.272133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.272521 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.272521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.272571 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.272882 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.272886 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.274099 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.275981 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.275982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.275983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.276153 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.276333 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.276334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.276334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.276374 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.276691 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.276692 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.276733 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.276801 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.276911 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260808 02:13:21.277743 30659 heartbeater.cc:347] Connected to a master server at 127.19.58.254:32909
I20260808 02:13:21.277820 30659 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.277904 30659 heartbeater.cc:511] Master 127.19.58.254:32909 requested a full tablet report, sending...
I20260808 02:13:21.278200 29011 ts_manager.cc:194] Re-registered known tserver with Master: 1d3360d0edc04a44b0260e36dcf59dfd (127.19.58.194:42639)
I20260808 02:13:21.278837 29011 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:34053
I20260808 02:13:21.283887 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:37345
--local_ip_for_outbound_sockets=127.19.58.195
--tserver_master_addrs=127.19.58.254:32909
--webserver_port=38725
--webserver_interface=127.19.58.195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab
--principal=oryx/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.58.254:32909,127.19.58.253:33461
--tserver_master_addrs=127.19.58.254:32909,127.19.58.253:33461,127.19.58.252:34863 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:21.364087 30667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:21.364289 30667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:21.364312 30667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:21.365471 30667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:21.365512 30667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:21.365537 30667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:21.366950 30667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab
--principal=oryx/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:37345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=38725
--tserver_master_addrs=127.19.58.254:32909,127.19.58.253:33461,127.19.58.252:34863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:21.367130 30667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:21.367303 30667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:21.372093 30682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:21.372093 30678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:21.372241 30667 server_base.cc:1034] running on GCE node
W20260808 02:13:21.372113 30675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:21.372475 30667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:21.372665 30667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:21.373795 30667 hybrid_clock.cc:648] HybridClock initialized: now 1786155201373783 us; error 29 us; skew 500 ppm
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:21.376228 30667 init.cc:377] Logged in from keytab as oryx/127.19.58.195@KRBTEST.COM (short username oryx)
I20260808 02:13:21.376631 30667 webserver.cc:460] Webserver started at http://127.19.58.195:38725/ using document root <none> and password file <none>
I20260808 02:13:21.376835 30667 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:21.376889 30667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:21.378123 30667 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:21.378643 30694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.378772 30667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.378849 30667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "1dc6ea81f1ba4e1083af64eddea9f858"
format_stamp: "Formatted at 2026-08-08 02:13:20 on dist-test-slave-1l3l"
server_key: "ee54b4a6363ea1a32ecd63fe7a8bc863"
server_key_iv: "c6f6f2b9724246268de82a3e981d58c6"
server_key_version: "encryptionkey@0"
I20260808 02:13:21.379146 30667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:21.403618 30667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:21.405419 30667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:21.405553 30667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:21.405755 30667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:21.406045 30667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:21.406080 30667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.406109 30667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:21.406132 30667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:21.411849 30667 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:37345
I20260808 02:13:21.411908 30861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:37345 every 8 connection(s)
I20260808 02:13:21.412204 30667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:21.419029 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 30667
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.252@KRBTEST.COM
Aug 08 02:13:21 dist-test-slave-1l3l krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155201, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.253@KRBTEST.COM
I20260808 02:13:21.423769 30665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.412896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:43649 (local address 127.19.58.252:34863)
0808 02:13:21.413126 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.413128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.413374 (+ 246us) server_negotiation.cc:407] Connection header received
0808 02:13:21.414171 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.414172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.414184 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.414226 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.414923 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.415685 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.416608 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.416858 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.416921 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.420087 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.420088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.420088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.420096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.421768 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.422258 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.422259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.422259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.422332 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.422631 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.422632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.422632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.422785 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.422851 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.423553 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.423641 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260808 02:13:21.424193 30868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.412881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34863 (local address 127.19.58.195:43649)
0808 02:13:21.413310 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.413318 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.414063 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.414285 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.414286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.414607 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.414791 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.414797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.415790 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.415791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.416524 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.416527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.416934 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.416935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.417065 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.417494 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.417502 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.420007 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.421871 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.421879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.421881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.422173 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.422428 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.422429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.422429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.422558 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.422892 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.422899 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.423044 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.423140 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.423304 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":4,"thread_start_us":92,"threads_started":1}
I20260808 02:13:21.424841 30862 heartbeater.cc:347] Connected to a master server at 127.19.58.252:34863
I20260808 02:13:21.424973 30862 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.425153 30862 heartbeater.cc:511] Master 127.19.58.252:34863 requested a full tablet report, sending...
I20260808 02:13:21.425529 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.412860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:33461 (local address 127.19.58.195:36415)
0808 02:13:21.413031 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.413065 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.414047 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.414278 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.414281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.414612 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.414791 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.414795 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.415759 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.415760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.416683 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.416686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.417149 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.417150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.417229 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.417759 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.417763 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.420666 (+ 2903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.423447 (+ 2781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.423448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.423449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.423663 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.424180 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.424180 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.424181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.424234 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.424749 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.424750 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.424791 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.424852 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.424942 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":33,"thread_start_us":102,"threads_started":1}
I20260808 02:13:21.425575 30184 ts_manager.cc:194] Registered new tserver with Master: 1dc6ea81f1ba4e1083af64eddea9f858 (127.19.58.195:37345)
I20260808 02:13:21.425655 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.412870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:36415 (local address 127.19.58.253:33461)
0808 02:13:21.412921 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.412923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.413147 (+ 224us) server_negotiation.cc:407] Connection header received
0808 02:13:21.414149 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.414150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.414164 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.414216 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.414875 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.415651 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.416800 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.417038 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.417098 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.420757 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.420759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.420759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.420767 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.422630 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.424058 (+ 1428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.424059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.424059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.424117 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.424540 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.424541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.424541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.424667 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.424723 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.425500 (+ 777us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.425572 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:13:21.426081 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.413235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:47553 (local address 127.19.58.254:32909)
0808 02:13:21.413284 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.413286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.413342 (+ 56us) server_negotiation.cc:407] Connection header received
0808 02:13:21.414169 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.414170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.414183 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.414216 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.414945 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.415462 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.416765 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.417024 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.417081 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.419210 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.419211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.419212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.419220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.423350 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.424298 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.424299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.424300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.424349 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.425004 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.425005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.425005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.425132 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.425195 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.425903 (+ 708us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.425983 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:13:21.426126 30865 heartbeater.cc:347] Connected to a master server at 127.19.58.253:33461
I20260808 02:13:21.426162 30865 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.426219 30865 heartbeater.cc:511] Master 127.19.58.253:33461 requested a full tablet report, sending...
I20260808 02:13:21.426398 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.413149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.195:47553)
0808 02:13:21.413274 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.413283 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.414063 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.414285 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.414286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.414628 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:21.414791 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.414797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.415569 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.415570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.416686 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.416688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.417091 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.417092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.417159 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.417603 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:21.417608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:21.419116 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:21.423776 (+ 4660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.423777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.423777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.423964 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.424421 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:21.424421 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:21.424421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:21.424473 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:21.425223 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:21.425223 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:21.425261 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.425319 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.425422 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":15,"thread_start_us":74,"threads_started":1}
I20260808 02:13:21.426702 30864 heartbeater.cc:347] Connected to a master server at 127.19.58.254:32909
I20260808 02:13:21.426754 30864 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:21.426756 29915 ts_manager.cc:194] Registered new tserver with Master: 1dc6ea81f1ba4e1083af64eddea9f858 (127.19.58.195:37345)
I20260808 02:13:21.426827 30864 heartbeater.cc:511] Master 127.19.58.254:32909 requested a full tablet report, sending...
I20260808 02:13:21.427066 29011 ts_manager.cc:194] Re-registered known tserver with Master: 1dc6ea81f1ba4e1083af64eddea9f858 (127.19.58.195:37345)
I20260808 02:13:21.427484 29011 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:47553
I20260808 02:13:21.430459 30877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.420395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56758 (local address 127.19.58.253:33461)
0808 02:13:21.421048 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.421050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.421059 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:21.421077 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.421078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.421089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.421490 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.422404 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.423083 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.425924 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.426161 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.426480 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.427680 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.427681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.427681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.427688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.429342 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.429737 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.429738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.429738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.429795 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.429994 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.429995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.429995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.430111 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.430196 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.430286 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.430371 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":54,"threads_started":1}
I20260808 02:13:21.431221 30880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.420668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59020 (local address 127.19.58.254:32909)
0808 02:13:21.421827 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.421829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.421836 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:21.421867 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.421867 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.421879 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.421948 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.422641 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.423412 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.426669 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.426948 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.426998 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.428421 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.428422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.428422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.428429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.430018 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.430375 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.430377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.430377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.430433 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.430623 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.430624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.430624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.430739 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.430823 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.431045 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.431131 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1138,"thread_start_us":54,"threads_started":1}
I20260808 02:13:21.431358 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.420908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36340 (local address 127.19.58.252:34863)
0808 02:13:21.420949 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.420951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.421072 (+ 121us) server_negotiation.cc:407] Connection header received
0808 02:13:21.421162 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.421163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.421172 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.421202 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.423185 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.423948 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.424883 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.425160 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.425235 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.428798 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.428799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.428799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.428806 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.430185 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.430591 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.430592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.430592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.430644 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.430870 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.430871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.430871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.430982 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.431075 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.431188 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.431282 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:13:21.432649 29011 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:21.434175 29011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:21.434795 30129 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: NON_VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: true } } }
I20260808 02:13:21.434894 30129 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:21.436249 30887 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7: attempting to promote NON_VOTER 3ed51ae392984dc281859d6a42d27856 to VOTER
I20260808 02:13:21.436419 30887 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } }
I20260808 02:13:21.436776 30200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 1 LEARNER]: Refusing update from remote peer b3aec6d749834e9c99e150a0e4ba7eb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 02:13:21.436888 29932 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 FOLLOWER]: Refusing update from remote peer b3aec6d749834e9c99e150a0e4ba7eb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 02:13:21.436990 30885 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 02:13:21.437079 30129 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 02:13:21.437762 30129 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3ed51ae392984dc281859d6a42d27856 (127.19.58.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:21.437934 29932 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3ed51ae392984dc281859d6a42d27856 (127.19.58.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:21.438804 30886 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:21.438912 30886 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:21.439015 30129 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:21.439096 30129 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:21.439085 30895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:21.439019 30200 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3ed51ae392984dc281859d6a42d27856 (127.19.58.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:21.439163 30895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:21.439174 30887 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:21.439244 30887 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:21.440109 30244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:21.440197 30244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:21.444144 30898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.439908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:42639 (local address 127.0.0.1:42454)
0808 02:13:21.440379 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.440390 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.440506 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.440771 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.440772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.440774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:21.440819 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.440820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.442144 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.442145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.442937 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.442940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.443924 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.443926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.443984 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.443988 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.444033 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.444072 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":32,"thread_start_us":80,"threads_started":1}
I20260808 02:13:21.444264 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.439897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:35443 (local address 127.0.0.1:43340)
0808 02:13:21.440034 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.440044 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.440087 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.440998 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.440999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.441000 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:21.441033 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.441034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.442200 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.442201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.443246 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.443248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.444041 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.444043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.444103 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.444106 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.444160 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.444200 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":19}
I20260808 02:13:21.444370 30900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.440174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.19.58.193:35443)
0808 02:13:21.440823 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.440827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.440836 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:21.440884 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.440886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.440900 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.441103 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:21.441164 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.442139 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.443317 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.443973 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.444032 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.444126 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.444177 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.444291 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":125,"threads_started":1}
I20260808 02:13:21.444389 30901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.440278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42454 (local address 127.19.58.194:42639)
0808 02:13:21.440467 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.440472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.440485 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:13:21.440609 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.440612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.440629 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.440701 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:21.440892 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.442060 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.443019 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.443728 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.443798 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.443887 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.444256 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.444300 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
I20260808 02:13:21.444700 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.439908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:37345 (local address 127.0.0.1:41976)
0808 02:13:21.440454 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.440463 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.440506 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.440706 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.440707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.440709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:21.440752 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.440753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.441993 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.441994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.443132 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.443135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.444122 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.444123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.444178 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.444181 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.444581 (+ 400us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.444636 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260808 02:13:21.445041 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.440103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.19.58.195:37345)
0808 02:13:21.440556 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.440560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.440568 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:21.440601 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.440603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.440618 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.440697 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:21.440831 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.441909 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.443214 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.443966 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.444028 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.444080 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.444931 (+ 851us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.444972 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":85,"threads_started":1}
I20260808 02:13:21.444962 30359 tablet_service.cc:1467] Processing CreateTablet for tablet dd9b4739f8ba48ec89e9a28d13474958 (DEFAULT_TABLE table=test-table [id=c03ddb74bb2a41bcbba6ca91585a191a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:21.445019 30563 tablet_service.cc:1467] Processing CreateTablet for tablet dd9b4739f8ba48ec89e9a28d13474958 (DEFAULT_TABLE table=test-table [id=c03ddb74bb2a41bcbba6ca91585a191a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:21.445258 30359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd9b4739f8ba48ec89e9a28d13474958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:21.445349 30563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd9b4739f8ba48ec89e9a28d13474958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:21.447991 30905 tablet_bootstrap.cc:492] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: Bootstrap starting.
I20260808 02:13:21.448282 30907 tablet_bootstrap.cc:492] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: Bootstrap starting.
I20260808 02:13:21.448292 30760 tablet_service.cc:1467] Processing CreateTablet for tablet dd9b4739f8ba48ec89e9a28d13474958 (DEFAULT_TABLE table=test-table [id=c03ddb74bb2a41bcbba6ca91585a191a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:21.448645 30905 tablet_bootstrap.cc:654] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:21.448655 30760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd9b4739f8ba48ec89e9a28d13474958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:21.448909 30907 tablet_bootstrap.cc:654] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:21.448923 30905 log.cc:824] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:21.449230 30907 log.cc:824] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:21.449602 30905 tablet_bootstrap.cc:492] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: No bootstrap required, opened a new log
I20260808 02:13:21.449679 30905 ts_tablet_manager.cc:1397] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:21.450088 30907 tablet_bootstrap.cc:492] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: No bootstrap required, opened a new log
I20260808 02:13:21.450225 30907 ts_tablet_manager.cc:1397] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:21.451510 30905 raft_consensus.cc:348] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } }
I20260808 02:13:21.451633 30905 raft_consensus.cc:374] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:21.451653 30905 raft_consensus.cc:729] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d3360d0edc04a44b0260e36dcf59dfd, State: Initialized, Role: FOLLOWER
I20260808 02:13:21.451798 30905 consensus_queue.cc:260] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } }
I20260808 02:13:21.451995 30913 tablet_bootstrap.cc:492] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: Bootstrap starting.
I20260808 02:13:21.452066 30905 ts_tablet_manager.cc:1428] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:13:21.452185 30659 heartbeater.cc:503] Master 127.19.58.254:32909 was elected leader, sending a full tablet report...
I20260808 02:13:21.452615 30913 tablet_bootstrap.cc:654] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:21.452943 30913 log.cc:824] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:21.453293 30907 raft_consensus.cc:348] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } }
I20260808 02:13:21.453500 30907 raft_consensus.cc:374] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:21.453591 30907 raft_consensus.cc:729] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bceb22edd6e34183a043a4744086e1d7, State: Initialized, Role: FOLLOWER
I20260808 02:13:21.453768 30907 consensus_queue.cc:260] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } }
I20260808 02:13:21.454041 30907 ts_tablet_manager.cc:1428] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 02:13:21.454126 30465 heartbeater.cc:503] Master 127.19.58.254:32909 was elected leader, sending a full tablet report...
I20260808 02:13:21.454106 30913 tablet_bootstrap.cc:492] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: No bootstrap required, opened a new log
I20260808 02:13:21.454236 30913 ts_tablet_manager.cc:1397] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:21.456336 30913 raft_consensus.cc:348] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } }
I20260808 02:13:21.456465 30913 raft_consensus.cc:374] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:21.456496 30913 raft_consensus.cc:729] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dc6ea81f1ba4e1083af64eddea9f858, State: Initialized, Role: FOLLOWER
I20260808 02:13:21.456604 30913 consensus_queue.cc:260] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } }
I20260808 02:13:21.456810 30913 ts_tablet_manager.cc:1428] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:13:21.456881 30864 heartbeater.cc:503] Master 127.19.58.254:32909 was elected leader, sending a full tablet report...
W20260808 02:13:21.505307 30661 tablet.cc:2369] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:21.583076 30920 raft_consensus.cc:482] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:21.583192 30920 raft_consensus.cc:504] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } }
I20260808 02:13:21.583515 30920 leader_election.cc:288] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bceb22edd6e34183a043a4744086e1d7 (127.19.58.193:35443), 1d3360d0edc04a44b0260e36dcf59dfd (127.19.58.194:42639)
I20260808 02:13:21.586241 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.583674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:35443 (local address 127.19.58.195:40165)
0808 02:13:21.583751 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.583762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.583802 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.583973 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.583974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.583976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:21.584053 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.584054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.584667 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.584668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.585517 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.585519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.586019 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.586020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.586090 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.586093 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.586141 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.586170 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":35}
I20260808 02:13:21.586266 30900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.583769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:40165 (local address 127.19.58.193:35443)
0808 02:13:21.583811 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.583813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.583822 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:21.583896 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.583897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.583906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.583958 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:21.584118 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.584600 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.585605 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.585952 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.586004 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.586012 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.586159 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.586197 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:13:21.586328 30901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.583782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:44161 (local address 127.19.58.194:42639)
0808 02:13:21.583824 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.583826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.583835 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:21.583936 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.583937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.583946 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.583997 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:21.584123 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.584610 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.585568 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.586019 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.586088 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.586097 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.586234 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.586269 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:13:21.586369 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.583745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:42639 (local address 127.19.58.195:44161)
0808 02:13:21.583794 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:21.583804 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:21.583845 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:21.584008 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:21.584009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:21.584010 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:21.584062 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.584063 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.584681 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.584682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.585492 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:21.585494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.586091 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.586092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:21.586164 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.586167 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:21.586218 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 02:13:21.586256 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":1}
I20260808 02:13:21.586664 30394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9b4739f8ba48ec89e9a28d13474958" candidate_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceb22edd6e34183a043a4744086e1d7" is_pre_election: true
I20260808 02:13:21.586670 30590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9b4739f8ba48ec89e9a28d13474958" candidate_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" is_pre_election: true
I20260808 02:13:21.586817 30394 raft_consensus.cc:2454] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dc6ea81f1ba4e1083af64eddea9f858 in term 0.
I20260808 02:13:21.586817 30590 raft_consensus.cc:2454] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dc6ea81f1ba4e1083af64eddea9f858 in term 0.
I20260808 02:13:21.587042 30695 leader_election.cc:302] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d3360d0edc04a44b0260e36dcf59dfd, 1dc6ea81f1ba4e1083af64eddea9f858; no voters:
I20260808 02:13:21.587225 30920 raft_consensus.cc:2781] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:21.587266 30920 raft_consensus.cc:482] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:21.587287 30920 raft_consensus.cc:3037] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:21.588018 30920 raft_consensus.cc:504] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } }
I20260808 02:13:21.588203 30920 leader_election.cc:288] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [CANDIDATE]: Term 1 election: Requested vote from peers bceb22edd6e34183a043a4744086e1d7 (127.19.58.193:35443), 1d3360d0edc04a44b0260e36dcf59dfd (127.19.58.194:42639)
I20260808 02:13:21.588306 30590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9b4739f8ba48ec89e9a28d13474958" candidate_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d3360d0edc04a44b0260e36dcf59dfd"
I20260808 02:13:21.588385 30590 raft_consensus.cc:3037] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:21.589054 30590 raft_consensus.cc:2454] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dc6ea81f1ba4e1083af64eddea9f858 in term 1.
I20260808 02:13:21.589155 30394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9b4739f8ba48ec89e9a28d13474958" candidate_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceb22edd6e34183a043a4744086e1d7"
I20260808 02:13:21.589202 30695 leader_election.cc:302] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d3360d0edc04a44b0260e36dcf59dfd, 1dc6ea81f1ba4e1083af64eddea9f858; no voters:
I20260808 02:13:21.589237 30394 raft_consensus.cc:3037] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:21.589274 30920 raft_consensus.cc:2781] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:21.589438 30920 raft_consensus.cc:686] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 1 LEADER]: Becoming Leader. State: Replica: 1dc6ea81f1ba4e1083af64eddea9f858, State: Running, Role: LEADER
I20260808 02:13:21.589530 30920 consensus_queue.cc:237] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } }
I20260808 02:13:21.590049 30394 raft_consensus.cc:2454] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dc6ea81f1ba4e1083af64eddea9f858 in term 1.
I20260808 02:13:21.590428 29011 catalog_manager.cc:5495] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dc6ea81f1ba4e1083af64eddea9f858 (127.19.58.195). New cstate: current_term: 1 leader_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } health_report { overall_health: UNKNOWN } } }
W20260808 02:13:21.615384 30470 tablet.cc:2369] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:21.621281 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.619086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41978 (local address 127.19.58.195:37345)
0808 02:13:21.619129 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.619131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.619139 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:21.619215 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.619216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.619231 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.619283 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:21.619412 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.619958 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.620707 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.620901 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.620948 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.621059 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:21.621150 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:21.621202 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.621216 (+ 14us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.621231 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:13:21.622965 30590 raft_consensus.cc:1261] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd [term 1 FOLLOWER]: Refusing update from remote peer 1dc6ea81f1ba4e1083af64eddea9f858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:21.623001 30394 raft_consensus.cc:1261] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7 [term 1 FOLLOWER]: Refusing update from remote peer 1dc6ea81f1ba4e1083af64eddea9f858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:21.623291 30921 consensus_queue.cc:1035] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:21.623415 30920 consensus_queue.cc:1035] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:21.625721 30924 mvcc.cc:205] Tried to move back new op lower bound from 7316091705845035008 to 7316091705711390720. Current Snapshot: MvccSnapshot[applied={T|T < 7316091705845035008}]
I20260808 02:13:21.629244 29011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:21.629326 29011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:21.630575 29011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3aec6d749834e9c99e150a0e4ba7eb7: Sending DeleteTablet for 3 replicas of tablet dd9b4739f8ba48ec89e9a28d13474958
I20260808 02:13:21.630865 30359 tablet_service.cc:1514] Processing DeleteTablet for tablet dd9b4739f8ba48ec89e9a28d13474958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:21 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:43340
I20260808 02:13:21.630968 30760 tablet_service.cc:1514] Processing DeleteTablet for tablet dd9b4739f8ba48ec89e9a28d13474958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:21 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:41976
I20260808 02:13:21.631276 30935 tablet_replica.cc:330] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: stopping tablet replica
I20260808 02:13:21.631345 30563 tablet_service.cc:1514] Processing DeleteTablet for tablet dd9b4739f8ba48ec89e9a28d13474958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:21 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:42454
I20260808 02:13:21.631435 30936 tablet_replica.cc:330] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: stopping tablet replica
I20260808 02:13:21.631536 30936 raft_consensus.cc:2229] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:21.631624 30936 raft_consensus.cc:2258] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:21.631775 30935 raft_consensus.cc:2229] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:21.631923 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 28966
I20260808 02:13:21.631945 30935 raft_consensus.cc:2258] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:21.634064 30938 tablet_replica.cc:330] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: stopping tablet replica
I20260808 02:13:21.634147 30938 raft_consensus.cc:2229] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:21.634218 30938 raft_consensus.cc:2258] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:21.640736 30938 ts_tablet_manager.cc:1905] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:21.642666 30935 ts_tablet_manager.cc:1905] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:21.643317 30938 ts_tablet_manager.cc:1918] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:21.643473 30938 log.cc:1196] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/dd9b4739f8ba48ec89e9a28d13474958
I20260808 02:13:21.643832 30938 ts_tablet_manager.cc:1939] T dd9b4739f8ba48ec89e9a28d13474958 P 1d3360d0edc04a44b0260e36dcf59dfd: Deleting consensus metadata
I20260808 02:13:21.644207 30935 ts_tablet_manager.cc:1918] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:21.644273 30935 log.cc:1196] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/dd9b4739f8ba48ec89e9a28d13474958
I20260808 02:13:21.644501 30935 ts_tablet_manager.cc:1939] T dd9b4739f8ba48ec89e9a28d13474958 P bceb22edd6e34183a043a4744086e1d7: Deleting consensus metadata
I20260808 02:13:21.645190 30936 ts_tablet_manager.cc:1905] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:21.646425 30936 ts_tablet_manager.cc:1918] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:21.646492 30936 log.cc:1196] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/dd9b4739f8ba48ec89e9a28d13474958
I20260808 02:13:21.646714 30936 ts_tablet_manager.cc:1939] T dd9b4739f8ba48ec89e9a28d13474958 P 1dc6ea81f1ba4e1083af64eddea9f858: Deleting consensus metadata
W20260808 02:13:21.648650 30497 connection.cc:535] server connection from 127.0.0.1:42454 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 02:13:21.648847 30289 connection.cc:535] server connection from 127.0.0.1:43340 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 02:13:21.648869 30699 connection.cc:535] server connection from 127.0.0.1:41976 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:13:21.657045 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.650434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36352 (local address 127.19.58.252:34863)
0808 02:13:21.650478 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.650480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.650489 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:21.650547 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.650548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.650558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.650604 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.651090 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.651823 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.652833 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.653090 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.653146 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.654341 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.654342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.654343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.654350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.655878 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.656318 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.656319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.656319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.656378 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.656542 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.656543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.656544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.656679 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.656773 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.656871 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.656947 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:13:21.657142 30877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:21.650223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56762 (local address 127.19.58.253:33461)
0808 02:13:21.650271 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:21.650273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:21.650458 (+ 185us) server_negotiation.cc:407] Connection header received
0808 02:13:21.650562 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:21.650563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:21.650578 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:21.650607 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:21.651292 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.651994 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.652680 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:21.652949 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:21.653010 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:21.654473 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:21.654474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:21.654475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:21.654481 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:21.656009 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.656369 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.656370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.656370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.656442 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:21.656603 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:21.656604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:21.656604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:21.656716 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:21.656803 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:21.656968 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 02:13:21.657050 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:13:22.627377 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.627124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.193:39047)
0808 02:13:22.627264 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.627331 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
W20260808 02:13:22.627683 30465 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:32909 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
I20260808 02:13:22.628084 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.627639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.194:52431)
0808 02:13:22.627829 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.627913 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
I20260808 02:13:22.628193 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:22.627880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.195:36397)
0808 02:13:22.628074 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:22.628149 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
W20260808 02:13:22.628319 30659 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:32909 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
W20260808 02:13:22.628355 30864 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:32909 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
I20260808 02:13:23.130683 30951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b3aec6d749834e9c99e150a0e4ba7eb7)
I20260808 02:13:23.130854 30951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } }
I20260808 02:13:23.131137 30952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b3aec6d749834e9c99e150a0e4ba7eb7)
I20260808 02:13:23.131160 30951 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b3aec6d749834e9c99e150a0e4ba7eb7 (127.19.58.254:32909), 520b53d52d04411aadfc1dbe16ecd079 (127.19.58.253:33461)
I20260808 02:13:23.131278 30952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } }
I20260808 02:13:23.131557 30952 leader_election.cc:288] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b3aec6d749834e9c99e150a0e4ba7eb7 (127.19.58.254:32909), 3ed51ae392984dc281859d6a42d27856 (127.19.58.252:34863)
I20260808 02:13:23.131580 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.131310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:59510)
0808 02:13:23.131468 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.131536 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":64,"threads_started":1}
W20260808 02:13:23.131759 30163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
I20260808 02:13:23.132385 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.132280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:59516)
0808 02:13:23.132331 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.132360 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 02:13:23.132511 30163 leader_election.cc:334] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b3aec6d749834e9c99e150a0e4ba7eb7 (127.19.58.254:32909): Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
I20260808 02:13:23.132166 30957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.131985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:59512)
0808 02:13:23.132083 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.132131 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":53,"threads_started":1}
W20260808 02:13:23.132961 29896 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
I20260808 02:13:23.133358 30957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.133265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:59530)
0808 02:13:23.133301 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.133334 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 02:13:23.133466 29896 leader_election.cc:334] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b3aec6d749834e9c99e150a0e4ba7eb7 (127.19.58.254:32909): Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
I20260808 02:13:23.135000 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.131331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:33461 (local address 127.0.0.1:56778)
0808 02:13:23.131611 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.131621 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:23.131691 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:23.131942 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:23.131943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:23.131946 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:23.132008 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.132011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.132999 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.133000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.134266 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.134269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.134773 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.134774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.134840 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.134844 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:23.134888 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:13:23.134933 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":44,"threads_started":1}
I20260808 02:13:23.134999 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.131455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.19.58.253:33461)
0808 02:13:23.131555 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:23.131557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:23.131681 (+ 124us) server_negotiation.cc:407] Connection header received
0808 02:13:23.131769 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:23.131770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:23.131780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:23.131855 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:23.132092 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.132858 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.134335 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.134703 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.134761 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.134772 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:23.134900 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 02:13:23.134935 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":74,"thread_start_us":47,"threads_started":1}
I20260808 02:13:23.135227 29932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ed51ae392984dc281859d6a42d27856" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "520b53d52d04411aadfc1dbe16ecd079" is_pre_election: true
I20260808 02:13:23.135334 29932 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ed51ae392984dc281859d6a42d27856 in term 1.
I20260808 02:13:23.135465 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.132330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34863 (local address 127.0.0.1:36368)
0808 02:13:23.132412 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.132420 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:23.132610 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:23.132755 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:23.132756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:23.132757 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:23.132796 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.132798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.133648 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.133649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.134643 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.134646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.135237 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.135238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.135283 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.135286 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:23.135355 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:13:23.135396 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":2,"thread_start_us":46,"threads_started":1}
I20260808 02:13:23.135480 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.132235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36368 (local address 127.19.58.252:34863)
0808 02:13:23.132514 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:23.132515 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:23.132524 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:23.132670 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:23.132671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:23.132680 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:23.132749 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:23.132861 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.133577 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.134723 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.135122 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.135196 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.135207 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:23.135372 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 02:13:23.135426 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":83,"threads_started":1}
I20260808 02:13:23.135501 30161 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ed51ae392984dc281859d6a42d27856, 520b53d52d04411aadfc1dbe16ecd079; no voters: b3aec6d749834e9c99e150a0e4ba7eb7
I20260808 02:13:23.135593 30951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 02:13:23.135659 30951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b3aec6d749834e9c99e150a0e4ba7eb7)
I20260808 02:13:23.135680 30951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:13:23.135735 30200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "520b53d52d04411aadfc1dbe16ecd079" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3ed51ae392984dc281859d6a42d27856" is_pre_election: true
I20260808 02:13:23.136492 30951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } }
I20260808 02:13:23.136622 30200 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 520b53d52d04411aadfc1dbe16ecd079 in current term 2: Already voted for candidate 3ed51ae392984dc281859d6a42d27856 in this term.
I20260808 02:13:23.136627 30951 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [CANDIDATE]: Term 2 election: Requested vote from peers b3aec6d749834e9c99e150a0e4ba7eb7 (127.19.58.254:32909), 520b53d52d04411aadfc1dbe16ecd079 (127.19.58.253:33461)
I20260808 02:13:23.136804 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.136693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:59552)
0808 02:13:23.136753 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.136783 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 02:13:23.136771 29932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ed51ae392984dc281859d6a42d27856" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "520b53d52d04411aadfc1dbe16ecd079"
I20260808 02:13:23.136834 29896 leader_election.cc:302] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 520b53d52d04411aadfc1dbe16ecd079; no voters: 3ed51ae392984dc281859d6a42d27856, b3aec6d749834e9c99e150a0e4ba7eb7
I20260808 02:13:23.136840 29932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:13:23.137094 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.137000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:59554)
0808 02:13:23.137044 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.137073 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 02:13:23.137173 30163 leader_election.cc:334] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b3aec6d749834e9c99e150a0e4ba7eb7 (127.19.58.254:32909): Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
I20260808 02:13:23.137547 29932 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ed51ae392984dc281859d6a42d27856 in term 2.
I20260808 02:13:23.137622 30952 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:13:23.137701 30161 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ed51ae392984dc281859d6a42d27856, 520b53d52d04411aadfc1dbe16ecd079; no voters: b3aec6d749834e9c99e150a0e4ba7eb7
I20260808 02:13:23.137791 30951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:13:23.137936 30951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [term 2 LEADER]: Becoming Leader. State: Replica: 3ed51ae392984dc281859d6a42d27856, State: Running, Role: LEADER
I20260808 02:13:23.138005 30951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } }
I20260808 02:13:23.138339 30967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed51ae392984dc281859d6a42d27856. Latest consensus state: current_term: 2 leader_uuid: "3ed51ae392984dc281859d6a42d27856" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:23.138458 30967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:23.138618 30969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:23.139009 30969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:23.139078 30969 catalog_manager.cc:1259] Loaded cluster ID: 1ccb96e17b9044c9b63ad047597abf1b
I20260808 02:13:23.139109 30969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:23.139283 30969 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:23.139375 30969 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Loaded TSK: 0
I20260808 02:13:23.139509 30969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:23.190022 30184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:23.191594 30184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:23.192597 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.192475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:59578)
0808 02:13:23.192526 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.192562 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 02:13:23.192619 29932 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [term 2 FOLLOWER]: Refusing update from remote peer 3ed51ae392984dc281859d6a42d27856: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 02:13:23.192879 30951 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [LEADER]: Connected to new peer: Peer: permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 02:13:23.192941 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.192856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.0.0.1:59582)
0808 02:13:23.192886 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.192913 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 02:13:23.193050 30163 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 -> Peer b3aec6d749834e9c99e150a0e4ba7eb7 (127.19.58.254:32909): Couldn't send request to peer b3aec6d749834e9c99e150a0e4ba7eb7. Status: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 02:13:23.193473 30952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed51ae392984dc281859d6a42d27856. Latest consensus state: current_term: 2 leader_uuid: "3ed51ae392984dc281859d6a42d27856" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:23.193559 30952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:23.193888 30972 mvcc.cc:205] Tried to move back new op lower bound from 7316091712275005440 to 7316091712053874688. Current Snapshot: MvccSnapshot[applied={T|T < 7316091712275005440}]
I20260808 02:13:23.194039 30967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3ed51ae392984dc281859d6a42d27856" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:23.194126 30967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:23.194170 30952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3ed51ae392984dc281859d6a42d27856" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b3aec6d749834e9c99e150a0e4ba7eb7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 32909 } } peers { permanent_uuid: "520b53d52d04411aadfc1dbe16ecd079" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 33461 } attrs { promote: false } } peers { permanent_uuid: "3ed51ae392984dc281859d6a42d27856" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34863 } attrs { promote: false } } }
I20260808 02:13:23.194258 30952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 520b53d52d04411aadfc1dbe16ecd079 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:13:23.194672 30973 mvcc.cc:205] Tried to move back new op lower bound from 7316091712275005440 to 7316091712053874688. Current Snapshot: MvccSnapshot[applied={T|T < 7316091712275005440 or (T in {7316091712275005440})}]
I20260808 02:13:23.198789 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.195678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:42639 (local address 127.0.0.1:42462)
0808 02:13:23.195732 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.195741 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:23.195774 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:23.196029 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:23.196030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:23.196032 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:23.196099 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.196101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.196867 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.196868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.197794 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.197797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.198582 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.198583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.198643 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.198647 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:23.198691 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:13:23.198729 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 02:13:23.198902 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.195785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:35443 (local address 127.0.0.1:43356)
0808 02:13:23.195866 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.195876 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:23.195926 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:23.196354 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:23.196355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:23.196357 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:23.196387 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.196388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.197299 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.197300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.197998 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.197999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.198701 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.198702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.198753 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.198756 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:23.198805 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 02:13:23.198839 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 02:13:23.198930 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.196099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.19.58.193:35443)
0808 02:13:23.196216 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:23.196217 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:23.196225 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:23.196249 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:23.196250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:23.196259 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:23.196445 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:23.196482 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.197191 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.198078 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.198623 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.198691 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.198701 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:23.198816 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 02:13:23.198852 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":65,"threads_started":1}
I20260808 02:13:23.199059 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.195774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42462 (local address 127.19.58.194:42639)
0808 02:13:23.195881 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:23.195883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:23.195890 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:23.195928 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:23.195929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:23.195940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:23.196203 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:23.196248 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.196797 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.197901 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.198517 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.198575 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.198589 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:23.198937 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 02:13:23.198985 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":65,"threads_started":1}
I20260808 02:13:23.199193 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.195825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:37345 (local address 127.0.0.1:41988)
0808 02:13:23.195976 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.195986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:23.196027 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:23.196267 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:23.196268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:23.196270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:23.196316 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.196318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.197234 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.197235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.198263 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:23.198266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.198981 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.198982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:23.199042 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.199046 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:23.199093 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 02:13:23.199134 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260808 02:13:23.199193 30359 tablet_service.cc:1467] Processing CreateTablet for tablet 81f0eb254c7e479e873862e9375c0b6d (DEFAULT_TABLE table=test-table [id=d0d2c0409a4f43f991114bec9340b187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:23.199324 30359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f0eb254c7e479e873862e9375c0b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:23.199441 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.195998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.19.58.195:37345)
0808 02:13:23.196109 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:23.196111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:23.196120 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:23.196145 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:23.196146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:23.196157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:23.196216 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:23.196390 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.197153 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.198325 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.198873 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.198933 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.198944 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:23.199326 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 02:13:23.199378 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":52,"threads_started":1}
I20260808 02:13:23.199474 30563 tablet_service.cc:1467] Processing CreateTablet for tablet 81f0eb254c7e479e873862e9375c0b6d (DEFAULT_TABLE table=test-table [id=d0d2c0409a4f43f991114bec9340b187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:23.199597 30563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f0eb254c7e479e873862e9375c0b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:23.200599 30984 tablet_bootstrap.cc:492] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7: Bootstrap starting.
I20260808 02:13:23.200652 30760 tablet_service.cc:1467] Processing CreateTablet for tablet 81f0eb254c7e479e873862e9375c0b6d (DEFAULT_TABLE table=test-table [id=d0d2c0409a4f43f991114bec9340b187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:23.200757 30760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f0eb254c7e479e873862e9375c0b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:23.200932 30984 tablet_bootstrap.cc:654] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:23.201512 30984 tablet_bootstrap.cc:492] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7: No bootstrap required, opened a new log
I20260808 02:13:23.201570 30984 ts_tablet_manager.cc:1397] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:23.201718 30984 raft_consensus.cc:348] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } }
I20260808 02:13:23.201787 30984 raft_consensus.cc:374] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:23.201805 30984 raft_consensus.cc:729] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bceb22edd6e34183a043a4744086e1d7, State: Initialized, Role: FOLLOWER
I20260808 02:13:23.201860 30984 consensus_queue.cc:260] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } }
I20260808 02:13:23.202152 30988 tablet_bootstrap.cc:492] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858: Bootstrap starting.
I20260808 02:13:23.202019 30984 ts_tablet_manager.cc:1428] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:23.202592 30988 tablet_bootstrap.cc:654] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:23.202603 30462 heartbeater.cc:503] Master 127.19.58.252:34863 was elected leader, sending a full tablet report...
I20260808 02:13:23.202626 30985 tablet_bootstrap.cc:492] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd: Bootstrap starting.
I20260808 02:13:23.202649 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.202434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.193:33517)
0808 02:13:23.202579 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.202615 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260808 02:13:23.202977 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.202869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.193:46699)
0808 02:13:23.202926 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.202954 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 02:13:23.203015 30985 tablet_bootstrap.cc:654] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:23.203545 30988 tablet_bootstrap.cc:492] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858: No bootstrap required, opened a new log
I20260808 02:13:23.203596 30988 ts_tablet_manager.cc:1397] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:23.203603 30985 tablet_bootstrap.cc:492] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd: No bootstrap required, opened a new log
I20260808 02:13:23.203640 30985 ts_tablet_manager.cc:1397] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:23.203733 30988 raft_consensus.cc:348] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } }
I20260808 02:13:23.203800 30988 raft_consensus.cc:374] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:23.203774 30985 raft_consensus.cc:348] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } }
I20260808 02:13:23.203819 30988 raft_consensus.cc:729] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dc6ea81f1ba4e1083af64eddea9f858, State: Initialized, Role: FOLLOWER
I20260808 02:13:23.203832 30985 raft_consensus.cc:374] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:23.203855 30985 raft_consensus.cc:729] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d3360d0edc04a44b0260e36dcf59dfd, State: Initialized, Role: FOLLOWER
I20260808 02:13:23.203861 30988 consensus_queue.cc:260] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } }
I20260808 02:13:23.203908 30985 consensus_queue.cc:260] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } }
I20260808 02:13:23.204022 30988 ts_tablet_manager.cc:1428] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:23.204176 30985 ts_tablet_manager.cc:1428] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:23.204407 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.204216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.195:46279)
0808 02:13:23.204330 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.204365 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":77,"threads_started":1}
I20260808 02:13:23.204546 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.204276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.194:49457)
0808 02:13:23.204458 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.204515 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":76,"threads_started":1}
I20260808 02:13:23.204905 30862 heartbeater.cc:503] Master 127.19.58.252:34863 was elected leader, sending a full tablet report...
I20260808 02:13:23.204974 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.204891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.195:44919)
0808 02:13:23.204927 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.204948 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 02:13:23.205421 30657 heartbeater.cc:503] Master 127.19.58.252:34863 was elected leader, sending a full tablet report...
I20260808 02:13:23.205477 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.205355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.194:46337)
0808 02:13:23.205399 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.205426 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 02:13:23.284294 30996 raft_consensus.cc:482] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:23.284409 30996 raft_consensus.cc:504] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } }
I20260808 02:13:23.284572 30996 leader_election.cc:288] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d3360d0edc04a44b0260e36dcf59dfd (127.19.58.194:42639), bceb22edd6e34183a043a4744086e1d7 (127.19.58.193:35443)
I20260808 02:13:23.284797 30394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f0eb254c7e479e873862e9375c0b6d" candidate_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceb22edd6e34183a043a4744086e1d7" is_pre_election: true
I20260808 02:13:23.284791 30590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f0eb254c7e479e873862e9375c0b6d" candidate_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" is_pre_election: true
I20260808 02:13:23.284909 30590 raft_consensus.cc:2454] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dc6ea81f1ba4e1083af64eddea9f858 in term 0.
I20260808 02:13:23.284909 30394 raft_consensus.cc:2454] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dc6ea81f1ba4e1083af64eddea9f858 in term 0.
I20260808 02:13:23.285097 30701 leader_election.cc:302] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dc6ea81f1ba4e1083af64eddea9f858, bceb22edd6e34183a043a4744086e1d7; no voters:
I20260808 02:13:23.285230 30996 raft_consensus.cc:2781] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:23.285288 30996 raft_consensus.cc:482] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:23.285306 30996 raft_consensus.cc:3037] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:23.286036 30996 raft_consensus.cc:504] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } }
I20260808 02:13:23.286154 30996 leader_election.cc:288] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [CANDIDATE]: Term 1 election: Requested vote from peers 1d3360d0edc04a44b0260e36dcf59dfd (127.19.58.194:42639), bceb22edd6e34183a043a4744086e1d7 (127.19.58.193:35443)
I20260808 02:13:23.286319 30590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f0eb254c7e479e873862e9375c0b6d" candidate_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d3360d0edc04a44b0260e36dcf59dfd"
I20260808 02:13:23.286305 30394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f0eb254c7e479e873862e9375c0b6d" candidate_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bceb22edd6e34183a043a4744086e1d7"
I20260808 02:13:23.286401 30590 raft_consensus.cc:3037] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:23.286420 30394 raft_consensus.cc:3037] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:23.287153 30590 raft_consensus.cc:2454] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dc6ea81f1ba4e1083af64eddea9f858 in term 1.
I20260808 02:13:23.287153 30394 raft_consensus.cc:2454] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dc6ea81f1ba4e1083af64eddea9f858 in term 1.
I20260808 02:13:23.287307 30695 leader_election.cc:302] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d3360d0edc04a44b0260e36dcf59dfd, 1dc6ea81f1ba4e1083af64eddea9f858; no voters:
I20260808 02:13:23.287405 30996 raft_consensus.cc:2781] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:23.287580 30996 raft_consensus.cc:686] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 1 LEADER]: Becoming Leader. State: Replica: 1dc6ea81f1ba4e1083af64eddea9f858, State: Running, Role: LEADER
I20260808 02:13:23.287639 30996 consensus_queue.cc:237] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } } peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } }
I20260808 02:13:23.287904 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.287807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.195:58053)
0808 02:13:23.287842 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.287873 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260808 02:13:23.288301 30183 catalog_manager.cc:5495] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dc6ea81f1ba4e1083af64eddea9f858 (127.19.58.195). New cstate: current_term: 1 leader_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dc6ea81f1ba4e1083af64eddea9f858" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:23.313284 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.311049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.19.58.195:37345)
0808 02:13:23.311100 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:23.311102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:23.311109 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:23.311129 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:23.311130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:23.311138 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:23.311186 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:23.311343 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.311882 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.312692 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:23.312908 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:23.312965 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:23.313090 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:23.313117 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:23.313153 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:23.313204 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 02:13:23.313224 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:13:23.314285 30394 raft_consensus.cc:1261] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7 [term 1 FOLLOWER]: Refusing update from remote peer 1dc6ea81f1ba4e1083af64eddea9f858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:23.314285 30590 raft_consensus.cc:1261] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd [term 1 FOLLOWER]: Refusing update from remote peer 1dc6ea81f1ba4e1083af64eddea9f858: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:23.314553 31004 consensus_queue.cc:1035] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bceb22edd6e34183a043a4744086e1d7" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:23.314677 30996 consensus_queue.cc:1035] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d3360d0edc04a44b0260e36dcf59dfd" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:23.315229 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.315131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.194:57225)
0808 02:13:23.315163 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.315196 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260808 02:13:23.315850 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.315662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.193:58771)
0808 02:13:23.315783 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.315815 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 02:13:23.316293 31014 mvcc.cc:205] Tried to move back new op lower bound from 7316091712773337088 to 7316091712666730496. Current Snapshot: MvccSnapshot[applied={T|T < 7316091712773337088}]
I20260808 02:13:23.316532 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.316305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.195:56653)
0808 02:13:23.316475 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.316504 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260808 02:13:23.316728 31001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.316618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.194:45687)
0808 02:13:23.316676 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.316699 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 02:13:23.317414 31000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.317293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.195:41913)
0808 02:13:23.317360 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.317384 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 02:13:23.317695 30183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:23.317777 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:23.317519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:32909 (local address 127.19.58.193:33099)
0808 02:13:23.317725 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.317753 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:32909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260808 02:13:23.317790 30183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:23.319043 30183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ed51ae392984dc281859d6a42d27856: Sending DeleteTablet for 3 replicas of tablet 81f0eb254c7e479e873862e9375c0b6d
I20260808 02:13:23.319317 30359 tablet_service.cc:1514] Processing DeleteTablet for tablet 81f0eb254c7e479e873862e9375c0b6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:23 UTC) from {username='oryx', principal='oryx/127.19.58.252@KRBTEST.COM'} at 127.0.0.1:43356
I20260808 02:13:23.319361 30563 tablet_service.cc:1514] Processing DeleteTablet for tablet 81f0eb254c7e479e873862e9375c0b6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:23 UTC) from {username='oryx', principal='oryx/127.19.58.252@KRBTEST.COM'} at 127.0.0.1:42462
I20260808 02:13:23.319335 30358 tablet_service.cc:1514] Processing DeleteTablet for tablet 81f0eb254c7e479e873862e9375c0b6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:23 UTC) from {username='oryx', principal='oryx/127.19.58.252@KRBTEST.COM'} at 127.0.0.1:43356
I20260808 02:13:23.319537 30760 tablet_service.cc:1514] Processing DeleteTablet for tablet 81f0eb254c7e479e873862e9375c0b6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:23 UTC) from {username='oryx', principal='oryx/127.19.58.252@KRBTEST.COM'} at 127.0.0.1:41988
I20260808 02:13:23.319566 31027 tablet_replica.cc:330] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd: stopping tablet replica
I20260808 02:13:23.319610 31027 raft_consensus.cc:2229] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:23.319659 31027 raft_consensus.cc:2258] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:23.319818 31025 tablet_replica.cc:330] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7: stopping tablet replica
I20260808 02:13:23.319913 31025 raft_consensus.cc:2229] T 81f0eb254c7e479e873862e9375c0b6d P bceb22edd6e34183a043a4744086e1d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:23.319931 31028 tablet_replica.cc:330] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858: stopping tablet replica
I20260808 02:13:23.319972 31028 raft_consensus.cc:2229] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:23.319998 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 30266
I20260808 02:13:23.320019 31028 raft_consensus.cc:2258] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:23.323925 31028 ts_tablet_manager.cc:1905] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:23.325562 31028 ts_tablet_manager.cc:1918] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:23.325635 31028 log.cc:1196] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/81f0eb254c7e479e873862e9375c0b6d
I20260808 02:13:23.325853 31028 ts_tablet_manager.cc:1939] T 81f0eb254c7e479e873862e9375c0b6d P 1dc6ea81f1ba4e1083af64eddea9f858: Deleting consensus metadata
I20260808 02:13:23.326372 30163 catalog_manager.cc:4841] TS 1dc6ea81f1ba4e1083af64eddea9f858 (127.19.58.195:37345): tablet 81f0eb254c7e479e873862e9375c0b6d (table test-table [id=d0d2c0409a4f43f991114bec9340b187]) successfully deleted
I20260808 02:13:23.329334 31027 ts_tablet_manager.cc:1905] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:23.330767 31027 ts_tablet_manager.cc:1918] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:23.330829 31027 log.cc:1196] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/81f0eb254c7e479e873862e9375c0b6d
I20260808 02:13:23.331045 31027 ts_tablet_manager.cc:1939] T 81f0eb254c7e479e873862e9375c0b6d P 1d3360d0edc04a44b0260e36dcf59dfd: Deleting consensus metadata
I20260808 02:13:23.331334 30161 catalog_manager.cc:4841] TS 1d3360d0edc04a44b0260e36dcf59dfd (127.19.58.194:42639): tablet 81f0eb254c7e479e873862e9375c0b6d (table test-table [id=d0d2c0409a4f43f991114bec9340b187]) successfully deleted
W20260808 02:13:23.334101 30981 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:23.333968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:35443 (local address 127.0.0.1:43368)
0808 02:13:23.334016 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:23.334045 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:35443: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 02:13:23.334260 30164 catalog_manager.cc:4584] TS bceb22edd6e34183a043a4744086e1d7 (127.19.58.193:35443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81f0eb254c7e479e873862e9375c0b6d: Network error: Client connection negotiation failed: client connection to 127.19.58.193:35443: connect: Connection reset by peer (error 104)
I20260808 02:13:23.334285 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 30475
I20260808 02:13:23.343255 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 30667
I20260808 02:13:23.353925 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 29719
I20260808 02:13:23.364990 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 29988
2026-08-08T02:13:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4849 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:23 dist-test-slave-1l3l krb5kdc[31031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:23 dist-test-slave-1l3l krb5kdc[31031](info): set up 2 sockets
Aug 08 02:13:23 dist-test-slave-1l3l krb5kdc[31031](info): commencing operation
krb5kdc: starting...
W20260808 02:13:24.498821 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:24 dist-test-slave-1l3l krb5kdc[31031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:24Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.254.keytab.
Entry for principal oryx/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.254.keytab.
I20260808 02:13:24.611920 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42019
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.254.keytab
--principal=oryx/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:42019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:24.691365 31049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:24.691550 31049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:24.691569 31049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:24.692651 31049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:24.692687 31049 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 02:13:24.692699 31049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:24.692710 31049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:24.692721 31049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:24.692735 31049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:24.694151 31049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:42019
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.254.keytab
--principal=oryx/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:24.694331 31049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:24.694492 31049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:13:24.699033 31049 server_base.cc:1034] running on GCE node
W20260808 02:13:24.699004 31059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:24.699036 31056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:24.698997 31057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:24.699540 31049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:24.699826 31049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:24.700963 31049 hybrid_clock.cc:648] HybridClock initialized: now 1786155204700952 us; error 33 us; skew 500 ppm
Aug 08 02:13:24 dist-test-slave-1l3l krb5kdc[31031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155204, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:24.703217 31049 init.cc:377] Logged in from keytab as oryx/127.19.58.254@KRBTEST.COM (short username oryx)
I20260808 02:13:24.703583 31049 webserver.cc:460] Webserver started at http://127.19.58.254:38335/ using document root <none> and password file <none>
I20260808 02:13:24.703799 31049 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:24.703845 31049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:24.703943 31049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:24.704862 31049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "37495041c5e24e2ca33ff4fb0a218f72"
format_stamp: "Formatted at 2026-08-08 02:13:24 on dist-test-slave-1l3l"
server_key: "d4c51989fc85e024474226587bb36aa7"
server_key_iv: "c66095b7a3765fb147eb4c23a304bd52"
server_key_version: "encryptionkey@0"
I20260808 02:13:24.705152 31049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "37495041c5e24e2ca33ff4fb0a218f72"
format_stamp: "Formatted at 2026-08-08 02:13:24 on dist-test-slave-1l3l"
server_key: "d4c51989fc85e024474226587bb36aa7"
server_key_iv: "c66095b7a3765fb147eb4c23a304bd52"
server_key_version: "encryptionkey@0"
I20260808 02:13:24.706292 31049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:13:24.706920 31069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:24.707077 31049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:24.707158 31049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "37495041c5e24e2ca33ff4fb0a218f72"
format_stamp: "Formatted at 2026-08-08 02:13:24 on dist-test-slave-1l3l"
server_key: "d4c51989fc85e024474226587bb36aa7"
server_key_iv: "c66095b7a3765fb147eb4c23a304bd52"
server_key_version: "encryptionkey@0"
I20260808 02:13:24.707211 31049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:24.724445 31049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:24.726044 31049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:24.726146 31049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:24.730463 31049 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:42019
I20260808 02:13:24.730525 31156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:42019 every 8 connection(s)
I20260808 02:13:24.730793 31049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:24.731420 31158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:24.733569 31158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72: Bootstrap starting.
I20260808 02:13:24.734120 31158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:24.734401 31158 log.cc:824] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:24.734995 31158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72: No bootstrap required, opened a new log
I20260808 02:13:24.736140 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 31049
I20260808 02:13:24.736214 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:24.736377 19691 external_mini_cluster.cc:1428] Setting key feef33a3d6afca0e6d680c725199408d
I20260808 02:13:24.737221 31158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37495041c5e24e2ca33ff4fb0a218f72" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42019 } }
I20260808 02:13:24.737347 31158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:24.737389 31158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37495041c5e24e2ca33ff4fb0a218f72, State: Initialized, Role: FOLLOWER
I20260808 02:13:24.737507 31158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37495041c5e24e2ca33ff4fb0a218f72" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42019 } }
I20260808 02:13:24.737573 31158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:24.737603 31158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:24.737648 31158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:24.738157 31158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37495041c5e24e2ca33ff4fb0a218f72" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42019 } }
I20260808 02:13:24.738248 31158 leader_election.cc:302] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37495041c5e24e2ca33ff4fb0a218f72; no voters:
I20260808 02:13:24.738401 31158 leader_election.cc:288] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:24.738436 31167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:24.738550 31167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [term 1 LEADER]: Becoming Leader. State: Replica: 37495041c5e24e2ca33ff4fb0a218f72, State: Running, Role: LEADER
I20260808 02:13:24.738647 31158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:24.738626 31167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37495041c5e24e2ca33ff4fb0a218f72" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42019 } }
I20260808 02:13:24.738982 31172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37495041c5e24e2ca33ff4fb0a218f72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37495041c5e24e2ca33ff4fb0a218f72" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42019 } } }
I20260808 02:13:24.739090 31172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:24.738969 31167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37495041c5e24e2ca33ff4fb0a218f72. Latest consensus state: current_term: 1 leader_uuid: "37495041c5e24e2ca33ff4fb0a218f72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37495041c5e24e2ca33ff4fb0a218f72" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42019 } } }
I20260808 02:13:24.739259 31167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72 [sys.catalog]: This master's current role is: LEADER
W20260808 02:13:24.740319 31203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:13:24.740375 31203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:13:24.740456 31183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:24.740635 31183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:24.743134 31183 catalog_manager.cc:1347] Generated new cluster ID: 97f1b329348944b791bfd070be812341
I20260808 02:13:24.743186 31183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:13:24 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:24.748132 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:24.736843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51054 (local address 127.19.58.254:42019)
0808 02:13:24.737029 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:24.737037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:24.737060 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:13:24.737186 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:24.737192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:24.737231 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:24.737342 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:24.738048 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:24.739290 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:24.740384 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:24.740962 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:24.741075 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:24.743827 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:24.743831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:24.743832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:24.743843 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:24.746449 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:24.746813 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:24.746816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:24.746816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:24.746941 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:24.747119 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:24.747120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:24.747120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:24.747409 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:24.747583 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:24.747650 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 02:13:24.747932 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":91,"threads_started":1}
I20260808 02:13:24.772630 31183 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:24.773176 31183 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:24.777546 31183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72: Generated new TSK 0
I20260808 02:13:24.777711 31183 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab.
Entry for principal oryx/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab.
I20260808 02:13:24.810405 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42019
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab
--principal=oryx/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:24.890026 31209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:24.890215 31209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:24.890239 31209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:24.891359 31209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:24.891402 31209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:24.891420 31209 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:13:24.891435 31209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:24.892984 31209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab
--principal=oryx/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.58.254:42019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:24.893193 31209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:24.893406 31209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:24.898170 31217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:24.898195 31218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:24.898169 31222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:24.898307 31209 server_base.cc:1034] running on GCE node
I20260808 02:13:24.898607 31209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:24.898844 31209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:24.899979 31209 hybrid_clock.cc:648] HybridClock initialized: now 1786155204899964 us; error 31 us; skew 500 ppm
Aug 08 02:13:24 dist-test-slave-1l3l krb5kdc[31031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155204, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:24.902338 31209 init.cc:377] Logged in from keytab as oryx/127.19.58.193@KRBTEST.COM (short username oryx)
I20260808 02:13:24.902716 31209 webserver.cc:460] Webserver started at http://127.19.58.193:42721/ using document root <none> and password file <none>
I20260808 02:13:24.902917 31209 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:24.902964 31209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:24.903090 31209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:24.904145 31209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "003b2fd3915a4bd29fdeb293c2a6352f"
format_stamp: "Formatted at 2026-08-08 02:13:24 on dist-test-slave-1l3l"
server_key: "47469416326c2575203e564fa923b0b0"
server_key_iv: "69938060be0370d998c6c660775a0a12"
server_key_version: "encryptionkey@0"
I20260808 02:13:24.904469 31209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "003b2fd3915a4bd29fdeb293c2a6352f"
format_stamp: "Formatted at 2026-08-08 02:13:24 on dist-test-slave-1l3l"
server_key: "47469416326c2575203e564fa923b0b0"
server_key_iv: "69938060be0370d998c6c660775a0a12"
server_key_version: "encryptionkey@0"
I20260808 02:13:24.905699 31209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:24.906334 31230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:24.906514 31209 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:24.906586 31209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "003b2fd3915a4bd29fdeb293c2a6352f"
format_stamp: "Formatted at 2026-08-08 02:13:24 on dist-test-slave-1l3l"
server_key: "47469416326c2575203e564fa923b0b0"
server_key_iv: "69938060be0370d998c6c660775a0a12"
server_key_version: "encryptionkey@0"
I20260808 02:13:24.906646 31209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:24.922837 31209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:24.924678 31209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:24.924809 31209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:24.925324 31209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:24.925372 31209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:24.925411 31209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:24.925437 31209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:24.931624 31209 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:39459
I20260808 02:13:24.931679 31396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:39459 every 8 connection(s)
Aug 08 02:13:24 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155204, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:24.932029 31209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:24.934558 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:24.925748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:52325 (local address 127.19.58.254:42019)
0808 02:13:24.925813 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:24.925816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:24.925988 (+ 172us) server_negotiation.cc:407] Connection header received
0808 02:13:24.926440 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:24.926442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:24.926460 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:24.926546 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:24.927294 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:24.928094 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:24.929563 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:24.929878 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:24.929960 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:24.932418 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:24.932419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:24.932420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:24.932427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:24.933525 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:24.933868 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:24.933868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:24.933869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:24.933931 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:24.934109 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:24.934110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:24.934110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:24.934201 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:24.934244 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:24.934431 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 02:13:24.934496 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:13:24.934690 31241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:24.925705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42019 (local address 127.19.58.193:52325)
0808 02:13:24.925849 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:24.925908 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:24.926347 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:24.926616 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:24.926620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:24.926927 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:24.927190 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:24.927196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:24.928212 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:24.928213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:24.929468 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:24.929472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:24.929977 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:24.929978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:24.930079 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:24.930563 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:24.930573 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:24.932307 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:24.933597 (+ 1290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:24.933604 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:24.933605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:24.933808 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:24.933982 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:24.933982 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:24.933983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:24.934068 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:24.934236 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:24.934239 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:24.934337 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 02:13:24.934418 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 02:13:24.934528 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260808 02:13:24.935184 31397 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42019
I20260808 02:13:24.935262 31397 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:24.935436 31397 heartbeater.cc:511] Master 127.19.58.254:42019 requested a full tablet report, sending...
I20260808 02:13:24.935567 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 31209
I20260808 02:13:24.935678 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:24.935930 19691 external_mini_cluster.cc:1428] Setting key 6d6cbe3c18460f5f0a147c6583099a9a
I20260808 02:13:24.935974 31094 ts_manager.cc:194] Registered new tserver with Master: 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459)
I20260808 02:13:24.936702 31094 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:52325
I20260808 02:13:24.942832 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:24.937030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:60527 (local address 127.19.58.254:42019)
0808 02:13:24.937064 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:24.937066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:24.937591 (+ 525us) server_negotiation.cc:407] Connection header received
0808 02:13:24.937616 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:24.937617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:24.937626 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:24.937688 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:24.937949 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:24.938539 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:24.939440 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:24.939654 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:24.939720 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:24.940576 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:24.940577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:24.940577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:24.940582 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:24.941862 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:24.942222 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:24.942223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:24.942223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:24.942274 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:24.942456 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:24.942456 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:24.942457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:24.942542 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:24.942583 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:24.942721 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 02:13:24.942775 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:13:24.942888 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:24.936999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42019 (local address 127.19.58.193:60527)
0808 02:13:24.937516 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:24.937526 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:24.937580 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:24.937689 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:24.937690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:24.937831 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:24.937893 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:24.937895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:24.938629 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:24.938629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:24.939358 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:24.939361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:24.939730 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:24.939731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:24.939793 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:24.940144 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:24.940149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:24.940505 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:24.941939 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:24.941940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:24.941941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:24.942155 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:24.942325 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:24.942326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:24.942326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:24.942398 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:24.942590 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:24.942591 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:24.942634 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:13:24.942703 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:13:24.942798 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":450,"threads_started":1}
WARNING: no policy specified for oryx/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab.
Entry for principal oryx/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab.
I20260808 02:13:24.960703 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42019
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab
--principal=oryx/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:25.041584 31410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:25.041775 31410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:25.041800 31410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:25.042950 31410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:25.042994 31410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:25.043012 31410 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:13:25.043040 31410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:25.044584 31410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab
--principal=oryx/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.58.254:42019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:25.044804 31410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:25.045008 31410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:25.049707 31418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:25.049775 31410 server_base.cc:1034] running on GCE node
W20260808 02:13:25.049686 31422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:25.049681 31419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:25.050123 31410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:25.050367 31410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:25.051514 31410 hybrid_clock.cc:648] HybridClock initialized: now 1786155205051499 us; error 30 us; skew 500 ppm
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155205, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:25.053778 31410 init.cc:377] Logged in from keytab as oryx/127.19.58.194@KRBTEST.COM (short username oryx)
I20260808 02:13:25.054198 31410 webserver.cc:460] Webserver started at http://127.19.58.194:38749/ using document root <none> and password file <none>
I20260808 02:13:25.054440 31410 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:25.054486 31410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:25.054603 31410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:25.055600 31410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "78651ec895b545b8b150080295c3dc86"
format_stamp: "Formatted at 2026-08-08 02:13:25 on dist-test-slave-1l3l"
server_key: "88d25011894d0c6027070b1295334059"
server_key_iv: "4388ddf570ee9846482d1d873cb4063e"
server_key_version: "encryptionkey@0"
I20260808 02:13:25.055964 31410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "78651ec895b545b8b150080295c3dc86"
format_stamp: "Formatted at 2026-08-08 02:13:25 on dist-test-slave-1l3l"
server_key: "88d25011894d0c6027070b1295334059"
server_key_iv: "4388ddf570ee9846482d1d873cb4063e"
server_key_version: "encryptionkey@0"
I20260808 02:13:25.057173 31410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:25.057825 31430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.057960 31410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:25.058046 31410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "78651ec895b545b8b150080295c3dc86"
format_stamp: "Formatted at 2026-08-08 02:13:25 on dist-test-slave-1l3l"
server_key: "88d25011894d0c6027070b1295334059"
server_key_iv: "4388ddf570ee9846482d1d873cb4063e"
server_key_version: "encryptionkey@0"
I20260808 02:13:25.058110 31410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:25.065654 31410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:25.067317 31410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:25.067431 31410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:25.067955 31410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:25.067996 31410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.068015 31410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:25.068027 31410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155205, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:25.074779 31410 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:44699
I20260808 02:13:25.074848 31599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:44699 every 8 connection(s)
I20260808 02:13:25.075114 31410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:25.077466 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.068316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:36461 (local address 127.19.58.254:42019)
0808 02:13:25.068384 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.068386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.068568 (+ 182us) server_negotiation.cc:407] Connection header received
0808 02:13:25.069175 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.069177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.069186 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.069244 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:25.070052 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.070660 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.071692 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.071933 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.071990 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.074987 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:25.074988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:25.074989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:25.074995 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:25.076330 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.076672 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.076673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.076673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.076723 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.076889 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.076889 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.076890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.076984 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:25.077032 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.077327 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.077402 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 02:13:25.077687 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.068289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42019 (local address 127.19.58.194:36461)
0808 02:13:25.068438 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.068483 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.069081 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.069320 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.069325 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.069664 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:25.069955 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.069962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.070760 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.070761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.071608 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.071611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.072005 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.072006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.072142 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.072607 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:25.072617 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:25.074907 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:25.076403 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:25.076409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:25.076411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:25.076609 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:25.076763 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:25.076764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:25.076764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:25.076847 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:25.077061 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:25.077069 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:25.077191 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.077298 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.077466 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260808 02:13:25.078159 31600 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42019
I20260808 02:13:25.078240 31600 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:25.078404 31600 heartbeater.cc:511] Master 127.19.58.254:42019 requested a full tablet report, sending...
I20260808 02:13:25.078774 31096 ts_manager.cc:194] Registered new tserver with Master: 78651ec895b545b8b150080295c3dc86 (127.19.58.194:44699)
I20260808 02:13:25.079178 31096 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:36461
I20260808 02:13:25.083841 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.079177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:43697 (local address 127.19.58.254:42019)
0808 02:13:25.079215 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.079217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.079329 (+ 112us) server_negotiation.cc:407] Connection header received
0808 02:13:25.079379 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.079380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.079389 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.079432 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:25.079747 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.080278 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.080922 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.081094 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.081138 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.081812 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:25.081813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:25.081814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:25.081821 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:25.083061 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.083289 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.083289 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.083290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.083337 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.083443 (+ 106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.083444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.083444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.083560 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:25.083604 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.083694 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.083783 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:13:25.083853 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.079104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42019 (local address 127.19.58.194:43697)
0808 02:13:25.079270 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.079279 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.079335 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.079438 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.079438 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.079609 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:25.079664 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.079666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.080356 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.080357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.080858 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.080860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.081142 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.081142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.081186 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.081504 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:25.081507 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:25.081742 (+ 235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:25.083118 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:25.083119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:25.083119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:25.083249 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:25.083370 (+ 121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:25.083370 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:25.083371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:25.083412 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:25.083602 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:25.083603 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:25.083634 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.083678 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.083785 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":62,"threads_started":1}
I20260808 02:13:25.084733 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 31410
I20260808 02:13:25.084851 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:25.085059 19691 external_mini_cluster.cc:1428] Setting key a2f87a3ba367264a0d2d2138bf196a73
WARNING: no policy specified for oryx/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab.
Entry for principal oryx/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab.
I20260808 02:13:25.110097 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42019
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab
--principal=oryx/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:25.191090 31616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:25.191282 31616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:25.191303 31616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:25.192492 31616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:25.192533 31616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:25.192552 31616 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 02:13:25.192571 31616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:25.194049 31616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab
--principal=oryx/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.58.254:42019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:25.194252 31616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:25.194461 31616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:25.199209 31625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:25.199246 31622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:25.199343 31616 server_base.cc:1034] running on GCE node
W20260808 02:13:25.199230 31623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:25.199635 31616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:25.199883 31616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:25.201031 31616 hybrid_clock.cc:648] HybridClock initialized: now 1786155205201017 us; error 40 us; skew 500 ppm
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155205, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:25.203603 31616 init.cc:377] Logged in from keytab as oryx/127.19.58.195@KRBTEST.COM (short username oryx)
I20260808 02:13:25.204075 31616 webserver.cc:460] Webserver started at http://127.19.58.195:45843/ using document root <none> and password file <none>
I20260808 02:13:25.204268 31616 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:25.204372 31616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:25.204509 31616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:25.205520 31616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "b54a68c96f614a5abc04bc297850f324"
format_stamp: "Formatted at 2026-08-08 02:13:25 on dist-test-slave-1l3l"
server_key: "ac3700d3b06e2bc405731445f998465c"
server_key_iv: "abf102368c5bed6641afb716176f5db3"
server_key_version: "encryptionkey@0"
I20260808 02:13:25.205854 31616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "b54a68c96f614a5abc04bc297850f324"
format_stamp: "Formatted at 2026-08-08 02:13:25 on dist-test-slave-1l3l"
server_key: "ac3700d3b06e2bc405731445f998465c"
server_key_iv: "abf102368c5bed6641afb716176f5db3"
server_key_version: "encryptionkey@0"
I20260808 02:13:25.207160 31616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:13:25.207923 31634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.208077 31616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:25.208159 31616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "b54a68c96f614a5abc04bc297850f324"
format_stamp: "Formatted at 2026-08-08 02:13:25 on dist-test-slave-1l3l"
server_key: "ac3700d3b06e2bc405731445f998465c"
server_key_iv: "abf102368c5bed6641afb716176f5db3"
server_key_version: "encryptionkey@0"
I20260808 02:13:25.208223 31616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:25.227185 31616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:25.229300 31616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:25.229440 31616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:25.229971 31616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:25.230013 31616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.230043 31616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:25.230067 31616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155205, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:25.236951 31616 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:35053
I20260808 02:13:25.237278 31616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:25.237601 31803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:35053 every 8 connection(s)
I20260808 02:13:25.238721 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.230345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:46023 (local address 127.19.58.254:42019)
0808 02:13:25.230417 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.230420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.230619 (+ 199us) server_negotiation.cc:407] Connection header received
0808 02:13:25.231093 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.231095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.231108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.231178 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:25.231824 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.232324 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.233298 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.233503 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.233558 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.236091 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:25.236092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:25.236092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:25.236100 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:25.237446 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.237847 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.237849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.237849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.237911 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.238160 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.238162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.238162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.238295 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:25.238369 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.238596 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.238657 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 02:13:25.238945 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.230318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42019 (local address 127.19.58.195:46023)
0808 02:13:25.230495 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.230534 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.231003 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.231242 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.231246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.231502 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:25.231735 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.231740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.232408 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.232409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.233214 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.233216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.233573 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.233574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.233692 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.234125 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:25.234133 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:25.236004 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:25.237524 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:25.237532 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:25.237534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:25.237780 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:25.237986 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:25.237987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:25.237987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:25.238098 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:25.238362 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:25.238365 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:25.238476 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.238584 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.238743 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
I20260808 02:13:25.239465 31804 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42019
I20260808 02:13:25.239555 31804 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:25.239749 31804 heartbeater.cc:511] Master 127.19.58.254:42019 requested a full tablet report, sending...
I20260808 02:13:25.240139 31094 ts_manager.cc:194] Registered new tserver with Master: b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053)
I20260808 02:13:25.240607 31094 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:46023
I20260808 02:13:25.245193 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.240711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:57241 (local address 127.19.58.254:42019)
0808 02:13:25.240763 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.240764 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.240771 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:25.240788 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.240789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.240796 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.240841 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:25.241144 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.241598 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.242265 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.242497 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.242551 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.243281 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:25.243282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:25.243283 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:25.243289 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:25.244257 (+ 968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.244540 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.244541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.244541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.244591 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.244750 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.244750 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.244751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.244851 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:25.244906 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.245040 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.245106 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 02:13:25.245205 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.240435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42019 (local address 127.19.58.195:57241)
0808 02:13:25.240545 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.240553 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.240597 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.240855 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.240857 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.241019 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:25.241074 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.241075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.241686 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.241687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.242191 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.242193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.242561 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.242562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.242629 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.242952 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:25.242956 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:25.243201 (+ 245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:25.244326 (+ 1125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:25.244327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:25.244327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:25.244481 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:25.244642 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:25.244643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:25.244643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:25.244700 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:25.244905 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:25.244906 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:25.244950 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.245010 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.245107 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":65,"threads_started":1}
I20260808 02:13:25.245617 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 31616
I20260808 02:13:25.245724 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:25.245940 19691 external_mini_cluster.cc:1428] Setting key 861d2af99a4401ee2f593e6fd3b26c76
I20260808 02:13:25.246809 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155205, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:13:25.267630 31166 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:25.262738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.19.58.254:42019)
0808 02:13:25.262785 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.262786 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.262796 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:13:25.262820 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.262821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.262829 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.262877 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:25.263412 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.263944 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.264624 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.264818 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.264872 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.267556 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51066: BlockingRecv error: recv got EOF from 127.0.0.1:51066 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155205, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:25.274227 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.268361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51074 (local address 127.19.58.254:42019)
0808 02:13:25.268413 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.268415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.268485 (+ 70us) server_negotiation.cc:407] Connection header received
0808 02:13:25.268524 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.268524 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.268532 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.268577 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:25.268972 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.269403 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.270058 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.270217 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.270255 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.272264 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:25.272265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:25.272265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:25.272272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:25.273343 (+ 1071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.273615 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.273616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.273617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.273663 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.273868 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.273868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.273868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.273957 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:25.274023 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.274117 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.274169 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 02:13:25.275537 31094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:25.277022 31094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:25.283092 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.279283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43608 (local address 127.19.58.194:44699)
0808 02:13:25.279446 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.279452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.279522 (+ 70us) server_negotiation.cc:407] Connection header received
0808 02:13:25.279589 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.279591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.279608 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.279895 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:25.279958 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.281017 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.282215 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.282694 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.282757 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.282831 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.282960 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.282998 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260808 02:13:25.283075 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.279058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44699 (local address 127.0.0.1:43608)
0808 02:13:25.279470 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.279478 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.279524 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.279748 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.279751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.279753 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:25.279813 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.279815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.281101 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.281102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.282146 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.282148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.282770 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.282771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.282858 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.282872 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.282939 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.282995 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":2,"thread_start_us":41,"threads_started":1}
I20260808 02:13:25.283080 31835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.279052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:35053 (local address 127.0.0.1:45092)
0808 02:13:25.279276 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.279316 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.279379 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.279796 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.279797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.279799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:25.279842 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.279843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.280786 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.280787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.281974 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.281977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.282863 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.282865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.282929 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.282933 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.282962 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.283004 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":57,"threads_started":1}
I20260808 02:13:25.283365 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.279202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.19.58.195:35053)
0808 02:13:25.279641 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.279644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.279653 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:25.279688 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.279690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.279725 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.279801 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:25.279958 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.280716 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.282057 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.282753 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.283121 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.283208 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.283236 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.283281 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":83,"threads_started":1}
I20260808 02:13:25.283798 31496 tablet_service.cc:1467] Processing CreateTablet for tablet b995ed29f26447aaa27f0e2141ce7372 (DEFAULT_TABLE table=test-table [id=164ce270a5384bbba11a8d4c5742c5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.283947 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.279242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:39459 (local address 127.0.0.1:41442)
0808 02:13:25.279594 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.279604 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.279663 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.280099 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.280100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.280102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:25.280140 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.280141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.281300 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.281301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.282208 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.282211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.283698 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.283700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.283788 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.283792 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.283849 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.283886 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":47,"threads_started":1}
I20260808 02:13:25.284147 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.279438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41442 (local address 127.19.58.193:39459)
0808 02:13:25.279613 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.279618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.279653 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:13:25.279940 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.279942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.279959 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.280030 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:25.280234 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.281232 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.282298 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.282991 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.283048 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.283115 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.283881 (+ 766us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.283931 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260808 02:13:25.284045 31704 tablet_service.cc:1467] Processing CreateTablet for tablet b995ed29f26447aaa27f0e2141ce7372 (DEFAULT_TABLE table=test-table [id=164ce270a5384bbba11a8d4c5742c5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.284181 31496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b995ed29f26447aaa27f0e2141ce7372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.284379 31704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b995ed29f26447aaa27f0e2141ce7372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.286898 31843 tablet_bootstrap.cc:492] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: Bootstrap starting.
I20260808 02:13:25.287101 31844 tablet_bootstrap.cc:492] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: Bootstrap starting.
I20260808 02:13:25.287608 31843 tablet_bootstrap.cc:654] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.287536 31294 tablet_service.cc:1467] Processing CreateTablet for tablet b995ed29f26447aaa27f0e2141ce7372 (DEFAULT_TABLE table=test-table [id=164ce270a5384bbba11a8d4c5742c5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.287680 31844 tablet_bootstrap.cc:654] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.287928 31294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b995ed29f26447aaa27f0e2141ce7372. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.287932 31843 log.cc:824] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:25.288070 31844 log.cc:824] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:25.288688 31843 tablet_bootstrap.cc:492] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: No bootstrap required, opened a new log
I20260808 02:13:25.288727 31844 tablet_bootstrap.cc:492] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: No bootstrap required, opened a new log
I20260808 02:13:25.288765 31843 ts_tablet_manager.cc:1397] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 02:13:25.288782 31844 ts_tablet_manager.cc:1397] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 02:13:25.290160 31844 raft_consensus.cc:348] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.290316 31844 raft_consensus.cc:374] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.290354 31844 raft_consensus.cc:729] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78651ec895b545b8b150080295c3dc86, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.290460 31844 consensus_queue.cc:260] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.290729 31600 heartbeater.cc:503] Master 127.19.58.254:42019 was elected leader, sending a full tablet report...
I20260808 02:13:25.290696 31843 raft_consensus.cc:348] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.290848 31843 raft_consensus.cc:374] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.290899 31843 raft_consensus.cc:729] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54a68c96f614a5abc04bc297850f324, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.291009 31843 consensus_queue.cc:260] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.290719 31844 ts_tablet_manager.cc:1428] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:13:25.291270 31843 ts_tablet_manager.cc:1428] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 02:13:25.291391 31804 heartbeater.cc:503] Master 127.19.58.254:42019 was elected leader, sending a full tablet report...
I20260808 02:13:25.291635 31849 tablet_bootstrap.cc:492] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: Bootstrap starting.
I20260808 02:13:25.292191 31849 tablet_bootstrap.cc:654] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.292420 31849 log.cc:824] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:25.292960 31849 tablet_bootstrap.cc:492] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: No bootstrap required, opened a new log
I20260808 02:13:25.293007 31849 ts_tablet_manager.cc:1397] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:25.294401 31849 raft_consensus.cc:348] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.294516 31849 raft_consensus.cc:374] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.294536 31849 raft_consensus.cc:729] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003b2fd3915a4bd29fdeb293c2a6352f, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.294612 31849 consensus_queue.cc:260] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.294788 31849 ts_tablet_manager.cc:1428] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:25.294838 31397 heartbeater.cc:503] Master 127.19.58.254:42019 was elected leader, sending a full tablet report...
W20260808 02:13:25.325775 31602 tablet.cc:2369] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:25.430294 31852 raft_consensus.cc:482] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:25.430423 31852 raft_consensus.cc:504] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.430786 31852 leader_election.cc:288] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053), 78651ec895b545b8b150080295c3dc86 (127.19.58.194:44699)
W20260808 02:13:25.432746 31398 tablet.cc:2369] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:25.433696 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.431046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:60509 (local address 127.19.58.194:44699)
0808 02:13:25.431103 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.431105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.431116 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:13:25.431139 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.431140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.431148 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.431195 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:25.431368 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.431978 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.433026 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.433375 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.433438 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.433447 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.433585 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.433625 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:13:25.433698 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.431068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:53467 (local address 127.19.58.195:35053)
0808 02:13:25.431122 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.431124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.431355 (+ 231us) server_negotiation.cc:407] Connection header received
0808 02:13:25.431390 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.431391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.431401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.431446 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:25.431591 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.432077 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.433026 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.433382 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.433442 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.433451 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.433586 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.433627 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 02:13:25.433725 31241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.430987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44699 (local address 127.19.58.193:60509)
0808 02:13:25.431030 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.431042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.431085 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.431216 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.431217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.431219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:25.431296 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.431298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.432049 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.432050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.432953 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.432956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.433448 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.433449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.433522 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.433526 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.433570 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.433611 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":1}
I20260808 02:13:25.433737 31853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.430994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:35053 (local address 127.19.58.193:53467)
0808 02:13:25.431285 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.431294 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.431346 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.431460 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.431461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.431463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:25.431533 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.431534 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.432143 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.432144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.432934 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.432937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.433449 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.433450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.433527 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.433530 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.433574 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.433614 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":139,"thread_start_us":96,"threads_started":1}
I20260808 02:13:25.434063 31530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b995ed29f26447aaa27f0e2141ce7372" candidate_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78651ec895b545b8b150080295c3dc86" is_pre_election: true
I20260808 02:13:25.434093 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b995ed29f26447aaa27f0e2141ce7372" candidate_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54a68c96f614a5abc04bc297850f324" is_pre_election: true
I20260808 02:13:25.434221 31733 raft_consensus.cc:2454] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 003b2fd3915a4bd29fdeb293c2a6352f in term 0.
I20260808 02:13:25.434221 31530 raft_consensus.cc:2454] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 003b2fd3915a4bd29fdeb293c2a6352f in term 0.
I20260808 02:13:25.434453 31231 leader_election.cc:302] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003b2fd3915a4bd29fdeb293c2a6352f, b54a68c96f614a5abc04bc297850f324; no voters:
I20260808 02:13:25.434585 31852 raft_consensus.cc:2781] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:25.434648 31852 raft_consensus.cc:482] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:25.434700 31852 raft_consensus.cc:3037] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.435982 31852 raft_consensus.cc:504] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.436113 31852 leader_election.cc:288] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [CANDIDATE]: Term 1 election: Requested vote from peers b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053), 78651ec895b545b8b150080295c3dc86 (127.19.58.194:44699)
I20260808 02:13:25.436321 31530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b995ed29f26447aaa27f0e2141ce7372" candidate_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78651ec895b545b8b150080295c3dc86"
I20260808 02:13:25.436328 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b995ed29f26447aaa27f0e2141ce7372" candidate_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54a68c96f614a5abc04bc297850f324"
I20260808 02:13:25.436399 31530 raft_consensus.cc:3037] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.436401 31733 raft_consensus.cc:3037] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.437093 31733 raft_consensus.cc:2454] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 003b2fd3915a4bd29fdeb293c2a6352f in term 1.
I20260808 02:13:25.437115 31530 raft_consensus.cc:2454] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 003b2fd3915a4bd29fdeb293c2a6352f in term 1.
I20260808 02:13:25.437263 31231 leader_election.cc:302] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003b2fd3915a4bd29fdeb293c2a6352f, b54a68c96f614a5abc04bc297850f324; no voters:
I20260808 02:13:25.437384 31852 raft_consensus.cc:2781] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:25.437541 31852 raft_consensus.cc:686] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 LEADER]: Becoming Leader. State: Replica: 003b2fd3915a4bd29fdeb293c2a6352f, State: Running, Role: LEADER
I20260808 02:13:25.437628 31852 consensus_queue.cc:237] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.438472 31096 catalog_manager.cc:5495] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f reported cstate change: term changed from 0 to 1, leader changed from <none> to 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193). New cstate: current_term: 1 leader_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:25.465323 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.463069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.19.58.193:39459)
0808 02:13:25.463129 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.463131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.463138 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:25.463173 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.463174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.463192 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.463250 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:25.463400 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.463927 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.464634 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.464828 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.464882 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.465040 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:25.465136 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:25.465201 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.465232 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.465252 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:13:25.466887 31530 raft_consensus.cc:1261] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Refusing update from remote peer 003b2fd3915a4bd29fdeb293c2a6352f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:25.466892 31733 raft_consensus.cc:1261] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Refusing update from remote peer 003b2fd3915a4bd29fdeb293c2a6352f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:25.467232 31852 consensus_queue.cc:1035] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:25.467365 31854 consensus_queue.cc:1035] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [LEADER]: Connected to new peer: Peer: permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:25.469523 31857 mvcc.cc:205] Tried to move back new op lower bound from 7316091721589895168 to 7316091721473142784. Current Snapshot: MvccSnapshot[applied={T|T < 7316091721589895168}]
I20260808 02:13:25.470057 31858 mvcc.cc:205] Tried to move back new op lower bound from 7316091721589895168 to 7316091721473142784. Current Snapshot: MvccSnapshot[applied={T|T < 7316091721589895168}]
I20260808 02:13:25.470278 31860 mvcc.cc:205] Tried to move back new op lower bound from 7316091721589895168 to 7316091721473142784. Current Snapshot: MvccSnapshot[applied={T|T < 7316091721589895168}]
I20260808 02:13:25.472144 31096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:25.472239 31096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:25.472958 31096 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72: Sending DeleteTablet for 3 replicas of tablet b995ed29f26447aaa27f0e2141ce7372
I20260808 02:13:25.473246 31704 tablet_service.cc:1514] Processing DeleteTablet for tablet b995ed29f26447aaa27f0e2141ce7372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:45092
I20260808 02:13:25.473299 31294 tablet_service.cc:1514] Processing DeleteTablet for tablet b995ed29f26447aaa27f0e2141ce7372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:41442
I20260808 02:13:25.473389 31496 tablet_service.cc:1514] Processing DeleteTablet for tablet b995ed29f26447aaa27f0e2141ce7372 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:43608
I20260808 02:13:25.473506 31872 tablet_replica.cc:330] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: stopping tablet replica
I20260808 02:13:25.473596 31872 raft_consensus.cc:2229] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:25.473711 31872 raft_consensus.cc:2258] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.473719 31096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:25.473862 31096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:25.474154 31872 ts_tablet_manager.cc:1905] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.475099 31294 tablet_service.cc:1467] Processing CreateTablet for tablet 70fc11cfffb84abcae74e292fec14338 (DEFAULT_TABLE table=test-table [id=1a57385533b5405c8a916ca666a074b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.475102 31704 tablet_service.cc:1467] Processing CreateTablet for tablet 70fc11cfffb84abcae74e292fec14338 (DEFAULT_TABLE table=test-table [id=1a57385533b5405c8a916ca666a074b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.475220 31294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70fc11cfffb84abcae74e292fec14338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.475225 31704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70fc11cfffb84abcae74e292fec14338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.475867 31872 ts_tablet_manager.cc:1918] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:25.475937 31872 log.cc:1196] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/b995ed29f26447aaa27f0e2141ce7372
I20260808 02:13:25.476193 31872 ts_tablet_manager.cc:1939] T b995ed29f26447aaa27f0e2141ce7372 P 003b2fd3915a4bd29fdeb293c2a6352f: Deleting consensus metadata
I20260808 02:13:25.476511 31849 tablet_bootstrap.cc:492] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f: Bootstrap starting.
I20260808 02:13:25.476557 31873 tablet_replica.cc:330] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: stopping tablet replica
I20260808 02:13:25.476642 31873 raft_consensus.cc:2229] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:25.476732 31873 raft_consensus.cc:2258] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.476869 31843 tablet_bootstrap.cc:492] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: Bootstrap starting.
I20260808 02:13:25.476895 31849 tablet_bootstrap.cc:654] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.477263 31843 tablet_bootstrap.cc:654] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.477339 31496 tablet_service.cc:1467] Processing CreateTablet for tablet 70fc11cfffb84abcae74e292fec14338 (DEFAULT_TABLE table=test-table [id=1a57385533b5405c8a916ca666a074b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.477479 31496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70fc11cfffb84abcae74e292fec14338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.477792 31843 tablet_bootstrap.cc:492] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: No bootstrap required, opened a new log
I20260808 02:13:25.477839 31843 ts_tablet_manager.cc:1397] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:25.477972 31843 raft_consensus.cc:348] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.478024 31843 raft_consensus.cc:374] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.478041 31843 raft_consensus.cc:729] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54a68c96f614a5abc04bc297850f324, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.478078 31843 consensus_queue.cc:260] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.478157 31843 ts_tablet_manager.cc:1428] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.478778 31844 tablet_bootstrap.cc:492] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86: Bootstrap starting.
I20260808 02:13:25.478988 31070 catalog_manager.cc:4841] TS 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459): tablet b995ed29f26447aaa27f0e2141ce7372 (table test-table [id=164ce270a5384bbba11a8d4c5742c5c7]) successfully deleted
I20260808 02:13:25.479168 31844 tablet_bootstrap.cc:654] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.479233 31849 tablet_bootstrap.cc:492] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f: No bootstrap required, opened a new log
I20260808 02:13:25.479282 31849 ts_tablet_manager.cc:1397] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 02:13:25.479425 31849 raft_consensus.cc:348] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.479502 31849 raft_consensus.cc:374] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.479523 31849 raft_consensus.cc:729] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003b2fd3915a4bd29fdeb293c2a6352f, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.479575 31849 consensus_queue.cc:260] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.479660 31849 ts_tablet_manager.cc:1428] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:25.479812 31844 tablet_bootstrap.cc:492] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86: No bootstrap required, opened a new log
I20260808 02:13:25.479857 31844 ts_tablet_manager.cc:1397] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:25.480006 31844 raft_consensus.cc:348] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.480080 31844 raft_consensus.cc:374] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.480099 31844 raft_consensus.cc:729] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78651ec895b545b8b150080295c3dc86, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.480141 31844 consensus_queue.cc:260] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.480230 31844 ts_tablet_manager.cc:1428] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.480501 31873 ts_tablet_manager.cc:1905] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.481907 31873 ts_tablet_manager.cc:1918] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:25.481979 31873 log.cc:1196] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/b995ed29f26447aaa27f0e2141ce7372
I20260808 02:13:25.482221 31873 ts_tablet_manager.cc:1939] T b995ed29f26447aaa27f0e2141ce7372 P b54a68c96f614a5abc04bc297850f324: Deleting consensus metadata
I20260808 02:13:25.482587 31070 catalog_manager.cc:4841] TS b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053): tablet b995ed29f26447aaa27f0e2141ce7372 (table test-table [id=164ce270a5384bbba11a8d4c5742c5c7]) successfully deleted
I20260808 02:13:25.483810 31875 tablet_replica.cc:330] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: stopping tablet replica
I20260808 02:13:25.483897 31875 raft_consensus.cc:2229] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:25.483991 31875 raft_consensus.cc:2258] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.484386 31875 ts_tablet_manager.cc:1905] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.485458 31875 ts_tablet_manager.cc:1918] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:25.485515 31875 log.cc:1196] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/b995ed29f26447aaa27f0e2141ce7372
I20260808 02:13:25.485703 31875 ts_tablet_manager.cc:1939] T b995ed29f26447aaa27f0e2141ce7372 P 78651ec895b545b8b150080295c3dc86: Deleting consensus metadata
I20260808 02:13:25.486034 31075 catalog_manager.cc:4841] TS 78651ec895b545b8b150080295c3dc86 (127.19.58.194:44699): tablet b995ed29f26447aaa27f0e2141ce7372 (table test-table [id=164ce270a5384bbba11a8d4c5742c5c7]) successfully deleted
W20260808 02:13:25.488143 31805 tablet.cc:2369] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:25.655416 31852 raft_consensus.cc:482] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:25.655530 31852 raft_consensus.cc:504] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.655694 31852 leader_election.cc:288] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78651ec895b545b8b150080295c3dc86 (127.19.58.194:44699), b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053)
I20260808 02:13:25.655979 31530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70fc11cfffb84abcae74e292fec14338" candidate_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78651ec895b545b8b150080295c3dc86" is_pre_election: true
I20260808 02:13:25.655977 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70fc11cfffb84abcae74e292fec14338" candidate_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54a68c96f614a5abc04bc297850f324" is_pre_election: true
I20260808 02:13:25.656075 31530 raft_consensus.cc:2454] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 003b2fd3915a4bd29fdeb293c2a6352f in term 0.
I20260808 02:13:25.656078 31733 raft_consensus.cc:2454] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 003b2fd3915a4bd29fdeb293c2a6352f in term 0.
I20260808 02:13:25.656212 31234 leader_election.cc:302] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003b2fd3915a4bd29fdeb293c2a6352f, 78651ec895b545b8b150080295c3dc86; no voters:
I20260808 02:13:25.656325 31852 raft_consensus.cc:2781] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:25.656378 31852 raft_consensus.cc:482] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:25.656410 31852 raft_consensus.cc:3037] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.657110 31852 raft_consensus.cc:504] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.657222 31852 leader_election.cc:288] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [CANDIDATE]: Term 1 election: Requested vote from peers 78651ec895b545b8b150080295c3dc86 (127.19.58.194:44699), b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053)
I20260808 02:13:25.657372 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70fc11cfffb84abcae74e292fec14338" candidate_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54a68c96f614a5abc04bc297850f324"
I20260808 02:13:25.657385 31530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70fc11cfffb84abcae74e292fec14338" candidate_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78651ec895b545b8b150080295c3dc86"
I20260808 02:13:25.657459 31530 raft_consensus.cc:3037] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.657454 31733 raft_consensus.cc:3037] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.658066 31530 raft_consensus.cc:2454] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 003b2fd3915a4bd29fdeb293c2a6352f in term 1.
I20260808 02:13:25.658077 31733 raft_consensus.cc:2454] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 003b2fd3915a4bd29fdeb293c2a6352f in term 1.
I20260808 02:13:25.658226 31231 leader_election.cc:302] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003b2fd3915a4bd29fdeb293c2a6352f, b54a68c96f614a5abc04bc297850f324; no voters:
I20260808 02:13:25.658317 31852 raft_consensus.cc:2781] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:25.658370 31852 raft_consensus.cc:686] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 LEADER]: Becoming Leader. State: Replica: 003b2fd3915a4bd29fdeb293c2a6352f, State: Running, Role: LEADER
I20260808 02:13:25.658430 31852 consensus_queue.cc:237] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.658969 31096 catalog_manager.cc:5495] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f reported cstate change: term changed from 0 to 1, leader changed from <none> to 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193). New cstate: current_term: 1 leader_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } health_report { overall_health: UNKNOWN } } }
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155204, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.254@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:25.734658 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.728384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42019 (local address 127.0.0.1:51088)
0808 02:13:25.728539 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.728550 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.728621 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.728748 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.728749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.729072 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:25.729120 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.729123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.729772 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.729773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.730285 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.730286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.730591 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.730592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.730639 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.731183 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:25.731187 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:25.732522 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:25.733709 (+ 1187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:25.733716 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:25.733718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:25.733896 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:25.734061 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:25.734061 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:25.734062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:25.734114 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:25.734308 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:25.734316 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:25.734361 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.734426 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.734548 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":3,"thread_start_us":81,"threads_started":1}
I20260808 02:13:25.734715 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.728460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.19.58.254:42019)
0808 02:13:25.728528 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.728530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.728609 (+ 79us) server_negotiation.cc:407] Connection header received
0808 02:13:25.728686 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.728687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.728697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.728737 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:25.729197 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.729703 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.730354 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.730534 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.730590 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.732596 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:25.732597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:25.732598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:25.732604 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:25.733642 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.733959 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.733960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.733960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.734011 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.734160 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.734161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.734162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.734259 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:25.734312 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.734454 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.734613 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":36}
I20260808 02:13:25.736555 31096 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:51088:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 02:13:25.736759 31096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:25.738062 31496 tablet_service.cc:1467] Processing CreateTablet for tablet cafdf4b77e2e4082a006f35d7711fbcf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bd3d488f9674133bd3d0a45643609c3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:13:25.738056 31704 tablet_service.cc:1467] Processing CreateTablet for tablet cafdf4b77e2e4082a006f35d7711fbcf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bd3d488f9674133bd3d0a45643609c3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:13:25.738076 31294 tablet_service.cc:1467] Processing CreateTablet for tablet cafdf4b77e2e4082a006f35d7711fbcf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bd3d488f9674133bd3d0a45643609c3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 02:13:25.738220 31704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cafdf4b77e2e4082a006f35d7711fbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.738220 31294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cafdf4b77e2e4082a006f35d7711fbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.738222 31496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cafdf4b77e2e4082a006f35d7711fbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.739640 31844 tablet_bootstrap.cc:492] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: Bootstrap starting.
I20260808 02:13:25.739727 31849 tablet_bootstrap.cc:492] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: Bootstrap starting.
I20260808 02:13:25.739734 31843 tablet_bootstrap.cc:492] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: Bootstrap starting.
I20260808 02:13:25.740060 31849 tablet_bootstrap.cc:654] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.740068 31844 tablet_bootstrap.cc:654] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.740093 31843 tablet_bootstrap.cc:654] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.740639 31849 tablet_bootstrap.cc:492] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: No bootstrap required, opened a new log
I20260808 02:13:25.740694 31849 ts_tablet_manager.cc:1397] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:25.740813 31849 raft_consensus.cc:348] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.740860 31849 raft_consensus.cc:374] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.740872 31849 raft_consensus.cc:729] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003b2fd3915a4bd29fdeb293c2a6352f, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.740868 31843 tablet_bootstrap.cc:492] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: No bootstrap required, opened a new log
I20260808 02:13:25.740906 31843 ts_tablet_manager.cc:1397] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:25.740909 31849 consensus_queue.cc:260] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.740976 31849 ts_tablet_manager.cc:1428] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.741017 31852 tablet_replica.cc:439] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } }
I20260808 02:13:25.741043 31843 raft_consensus.cc:348] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.741086 31852 tablet_replica.cc:442] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:25.741106 31843 raft_consensus.cc:374] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.741127 31843 raft_consensus.cc:729] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54a68c96f614a5abc04bc297850f324, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.741169 31843 consensus_queue.cc:260] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.741255 31843 ts_tablet_manager.cc:1428] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.741325 31844 tablet_bootstrap.cc:492] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: No bootstrap required, opened a new log
I20260808 02:13:25.741362 31844 ts_tablet_manager.cc:1397] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:25.741304 31848 tablet_replica.cc:439] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } }
I20260808 02:13:25.741397 31848 tablet_replica.cc:442] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:25.741501 31844 raft_consensus.cc:348] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.741569 31844 raft_consensus.cc:374] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.741590 31844 raft_consensus.cc:729] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78651ec895b545b8b150080295c3dc86, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.741631 31844 consensus_queue.cc:260] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.741714 31844 ts_tablet_manager.cc:1428] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.741748 31847 tablet_replica.cc:439] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } }
I20260808 02:13:25.741817 31847 tablet_replica.cc:442] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:25.831056 31847 raft_consensus.cc:482] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:25.831154 31847 raft_consensus.cc:504] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.831503 31847 leader_election.cc:288] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459), b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053)
I20260808 02:13:25.834477 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.831750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:51277 (local address 127.19.58.193:39459)
0808 02:13:25.831805 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.831807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.831954 (+ 147us) server_negotiation.cc:407] Connection header received
0808 02:13:25.831986 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.831987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.832000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.832066 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:25.832246 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.832799 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.833803 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.834166 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.834230 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.834238 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.834360 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.834408 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:13:25.834486 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.831978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:60479 (local address 127.19.58.195:35053)
0808 02:13:25.832033 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.832035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.832156 (+ 121us) server_negotiation.cc:407] Connection header received
0808 02:13:25.832203 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.832204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.832214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.832265 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:25.832397 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.832843 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.833792 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.834145 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.834199 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.834208 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.834359 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.834421 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:13:25.834507 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.831915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:35053 (local address 127.19.58.194:60479)
0808 02:13:25.832065 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.832095 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.832151 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.832268 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.832269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.832270 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:25.832339 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.832341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.832922 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.832924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.833707 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.833709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.834216 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.834217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.834289 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.834293 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.834338 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.834376 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":12,"thread_start_us":67,"threads_started":1}
I20260808 02:13:25.834559 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.831673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:39459 (local address 127.19.58.194:51277)
0808 02:13:25.831826 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.831838 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.831924 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.832098 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.832099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.832101 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:25.832177 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.832179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.832871 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.832872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.833715 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.833718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.834240 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.834241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.834298 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.834302 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.834349 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.834390 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":43,"thread_start_us":99,"threads_started":1}
I20260808 02:13:25.834782 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cafdf4b77e2e4082a006f35d7711fbcf" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54a68c96f614a5abc04bc297850f324" is_pre_election: true
I20260808 02:13:25.834870 31733 raft_consensus.cc:2454] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 0.
I20260808 02:13:25.834933 31326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cafdf4b77e2e4082a006f35d7711fbcf" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" is_pre_election: true
I20260808 02:13:25.835035 31431 leader_election.cc:302] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78651ec895b545b8b150080295c3dc86, b54a68c96f614a5abc04bc297850f324; no voters:
I20260808 02:13:25.835093 31326 raft_consensus.cc:2454] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 0.
I20260808 02:13:25.835147 31847 raft_consensus.cc:2781] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:25.835179 31847 raft_consensus.cc:482] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:25.835197 31847 raft_consensus.cc:3037] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.835996 31847 raft_consensus.cc:504] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.836107 31847 leader_election.cc:288] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 election: Requested vote from peers 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459), b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053)
I20260808 02:13:25.836268 31326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cafdf4b77e2e4082a006f35d7711fbcf" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003b2fd3915a4bd29fdeb293c2a6352f"
I20260808 02:13:25.836285 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cafdf4b77e2e4082a006f35d7711fbcf" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54a68c96f614a5abc04bc297850f324"
I20260808 02:13:25.836334 31326 raft_consensus.cc:3037] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.836349 31733 raft_consensus.cc:3037] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.836921 31733 raft_consensus.cc:2454] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 1.
I20260808 02:13:25.836974 31326 raft_consensus.cc:2454] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 1.
I20260808 02:13:25.837095 31431 leader_election.cc:302] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78651ec895b545b8b150080295c3dc86, b54a68c96f614a5abc04bc297850f324; no voters:
I20260808 02:13:25.837183 31847 raft_consensus.cc:2781] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:25.837288 31847 raft_consensus.cc:686] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [term 1 LEADER]: Becoming Leader. State: Replica: 78651ec895b545b8b150080295c3dc86, State: Running, Role: LEADER
I20260808 02:13:25.837357 31847 consensus_queue.cc:237] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } }
I20260808 02:13:25.837671 31894 tablet_replica.cc:439] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: TxnStatusTablet state changed. Reason: New leader 78651ec895b545b8b150080295c3dc86. Latest consensus state: current_term: 1 leader_uuid: "78651ec895b545b8b150080295c3dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } }
I20260808 02:13:25.837774 31894 tablet_replica.cc:442] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:13:25.837941 31900 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 02:13:25.838047 31900 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 02:13:25.838039 31096 catalog_manager.cc:5495] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78651ec895b545b8b150080295c3dc86 (127.19.58.194). New cstate: current_term: 1 leader_uuid: "78651ec895b545b8b150080295c3dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } health_report { overall_health: HEALTHY } } }
I20260808 02:13:25.857712 31886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.855013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44699 (local address 127.0.0.1:43620)
0808 02:13:25.855086 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.855099 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.855147 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.855319 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.855320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.855321 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:13:25.855367 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.855369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.856288 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.856289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.856846 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.856848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.857272 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.857272 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.857311 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.857326 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:13:25.857572 (+ 246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:13:25.857577 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.857612 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.857653 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 02:13:25.857731 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.855080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.19.58.194:44699)
0808 02:13:25.855129 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.855131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.855141 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:13:25.855223 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.855224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.855243 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.855319 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:25.855427 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.856209 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.856915 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.857209 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.857277 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.857394 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:25.857513 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:25.857581 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.857617 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.857637 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 02:13:25.858745 31733 raft_consensus.cc:1261] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Refusing update from remote peer 78651ec895b545b8b150080295c3dc86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:25.858843 31326 raft_consensus.cc:1261] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Refusing update from remote peer 78651ec895b545b8b150080295c3dc86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:25.859161 31847 consensus_queue.cc:1035] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:25.859292 31894 consensus_queue.cc:1035] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:25.859887 31848 tablet_replica.cc:439] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: TxnStatusTablet state changed. Reason: New leader 78651ec895b545b8b150080295c3dc86. Latest consensus state: current_term: 1 leader_uuid: "78651ec895b545b8b150080295c3dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } }
I20260808 02:13:25.859987 31848 tablet_replica.cc:442] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:25.860442 31894 tablet_replica.cc:439] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78651ec895b545b8b150080295c3dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } }
I20260808 02:13:25.860450 31852 tablet_replica.cc:439] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: TxnStatusTablet state changed. Reason: New leader 78651ec895b545b8b150080295c3dc86. Latest consensus state: current_term: 1 leader_uuid: "78651ec895b545b8b150080295c3dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } }
I20260808 02:13:25.860515 31894 tablet_replica.cc:442] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:13:25.860535 31852 tablet_replica.cc:442] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:25.861263 31848 tablet_replica.cc:439] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78651ec895b545b8b150080295c3dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } }
I20260808 02:13:25.861335 31848 tablet_replica.cc:442] T cafdf4b77e2e4082a006f35d7711fbcf P b54a68c96f614a5abc04bc297850f324: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:25.862222 31847 tablet_replica.cc:439] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78651ec895b545b8b150080295c3dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } }
I20260808 02:13:25.862299 31847 tablet_replica.cc:442] T cafdf4b77e2e4082a006f35d7711fbcf P 78651ec895b545b8b150080295c3dc86: This TxnStatusTablet replica's current role is: LEADER
I20260808 02:13:25.862798 31852 tablet_replica.cc:439] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78651ec895b545b8b150080295c3dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } }
I20260808 02:13:25.862891 31852 tablet_replica.cc:442] T cafdf4b77e2e4082a006f35d7711fbcf P 003b2fd3915a4bd29fdeb293c2a6352f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 02:13:25.866669 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.864315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:33651 (local address 127.19.58.194:44699)
0808 02:13:25.864360 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.864362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.864396 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:13:25.864463 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.864464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.864471 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.864512 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:25.864614 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.865037 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.866133 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.866307 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.866356 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.866508 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:25.866530 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:25.866561 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.866613 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.866626 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:13:25.866735 31912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.864192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44699 (local address 127.19.58.193:33651)
0808 02:13:25.864336 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.864347 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.864411 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.864516 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.864517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.864518 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:13:25.864563 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.864565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.865092 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.865093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.866051 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.866055 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.866366 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.866366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.866426 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.866442 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:13:25.866565 (+ 123us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:13:25.866567 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.866611 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.866657 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260808 02:13:25.869083 31733 raft_consensus.cc:1261] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Refusing update from remote peer 003b2fd3915a4bd29fdeb293c2a6352f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:25.869215 31530 raft_consensus.cc:1261] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Refusing update from remote peer 003b2fd3915a4bd29fdeb293c2a6352f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:25.869278 31852 consensus_queue.cc:1035] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:25.869400 31854 consensus_queue.cc:1035] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [LEADER]: Connected to new peer: Peer: permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:25.877343 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.874655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:38695 (local address 127.19.58.193:39459)
0808 02:13:25.874689 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.874691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.874814 (+ 123us) server_negotiation.cc:407] Connection header received
0808 02:13:25.874868 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.874868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.874885 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.874934 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:25.875073 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.875872 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.876576 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.876882 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.876945 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.877067 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:25.877119 (+ 52us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:25.877159 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.877251 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.877275 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 02:13:25.877369 31927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.874614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:39459 (local address 127.19.58.194:38695)
0808 02:13:25.874749 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.874759 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.874820 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.874951 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.874952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.874953 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:13:25.875018 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.875019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.875947 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.875948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.876509 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.876511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.876955 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.876955 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.876995 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.877008 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:13:25.877171 (+ 163us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:13:25.877174 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.877239 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.877280 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260808 02:13:25.883951 31094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:25.884035 31094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:25.884608 31094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72: Sending DeleteTablet for 3 replicas of tablet 70fc11cfffb84abcae74e292fec14338
I20260808 02:13:25.884882 31704 tablet_service.cc:1514] Processing DeleteTablet for tablet 70fc11cfffb84abcae74e292fec14338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:45092
I20260808 02:13:25.884927 31294 tablet_service.cc:1514] Processing DeleteTablet for tablet 70fc11cfffb84abcae74e292fec14338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:41442
I20260808 02:13:25.884989 31873 tablet_replica.cc:330] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: stopping tablet replica
I20260808 02:13:25.885006 31872 tablet_replica.cc:330] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f: stopping tablet replica
I20260808 02:13:25.885048 31872 raft_consensus.cc:2229] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:25.885047 31873 raft_consensus.cc:2229] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:25.885087 31873 raft_consensus.cc:2258] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.885102 31872 raft_consensus.cc:2258] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.885371 31872 ts_tablet_manager.cc:1905] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.885497 31873 ts_tablet_manager.cc:1905] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.887095 31872 ts_tablet_manager.cc:1918] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:25.887151 31872 log.cc:1196] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/70fc11cfffb84abcae74e292fec14338
I20260808 02:13:25.887331 31872 ts_tablet_manager.cc:1939] T 70fc11cfffb84abcae74e292fec14338 P 003b2fd3915a4bd29fdeb293c2a6352f: Deleting consensus metadata
I20260808 02:13:25.889921 31496 tablet_service.cc:1514] Processing DeleteTablet for tablet 70fc11cfffb84abcae74e292fec14338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:43608
I20260808 02:13:25.889931 31070 catalog_manager.cc:4841] TS 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459): tablet 70fc11cfffb84abcae74e292fec14338 (table test-table [id=1a57385533b5405c8a916ca666a074b9]) successfully deleted
I20260808 02:13:25.890017 31875 tablet_replica.cc:330] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86: stopping tablet replica
I20260808 02:13:25.890071 31875 raft_consensus.cc:2229] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:25.890107 31875 raft_consensus.cc:2258] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.890259 31873 ts_tablet_manager.cc:1918] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:25.890321 31873 log.cc:1196] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/70fc11cfffb84abcae74e292fec14338
I20260808 02:13:25.890384 31875 ts_tablet_manager.cc:1905] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.890522 31873 ts_tablet_manager.cc:1939] T 70fc11cfffb84abcae74e292fec14338 P b54a68c96f614a5abc04bc297850f324: Deleting consensus metadata
I20260808 02:13:25.890801 31070 catalog_manager.cc:4841] TS b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053): tablet 70fc11cfffb84abcae74e292fec14338 (table test-table [id=1a57385533b5405c8a916ca666a074b9]) successfully deleted
I20260808 02:13:25.891830 31875 ts_tablet_manager.cc:1918] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:25.891901 31875 log.cc:1196] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/70fc11cfffb84abcae74e292fec14338
I20260808 02:13:25.892084 31875 ts_tablet_manager.cc:1939] T 70fc11cfffb84abcae74e292fec14338 P 78651ec895b545b8b150080295c3dc86: Deleting consensus metadata
I20260808 02:13:25.892445 31075 catalog_manager.cc:4841] TS 78651ec895b545b8b150080295c3dc86 (127.19.58.194:44699): tablet 70fc11cfffb84abcae74e292fec14338 (table test-table [id=1a57385533b5405c8a916ca666a074b9]) successfully deleted
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 02:13:25.909970 31166 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:25.904351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.19.58.254:42019)
0808 02:13:25.904401 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.904404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.904536 (+ 132us) server_negotiation.cc:407] Connection header received
0808 02:13:25.904571 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.904572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.904584 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.904655 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:25.905199 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.906017 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.906728 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.907028 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.907096 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.909910 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51096: BlockingRecv error: recv got EOF from 127.0.0.1:51096 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 02:13:25 dist-test-slave-1l3l krb5kdc[31031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM
I20260808 02:13:25.917505 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.910793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.19.58.254:42019)
0808 02:13:25.910860 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.910861 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.910868 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:25.910886 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.910887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.910895 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.910941 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:25.911390 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.911990 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.912661 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.912855 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.912908 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.915112 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:25.915113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:25.915114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:25.915119 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:25.916416 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.916786 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.916787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.916787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.916832 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:25.917070 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:25.917071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:25.917072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:25.917169 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:25.917240 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.917378 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.917446 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:13:25.919081 31094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:25.919255 31094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:25.920661 31294 tablet_service.cc:1467] Processing CreateTablet for tablet baab5eb0b82f45f38196299b983c55b8 (DEFAULT_TABLE table=test-table [id=6a3c7b81c9b64105b76595dd9f16f049]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.920672 31496 tablet_service.cc:1467] Processing CreateTablet for tablet baab5eb0b82f45f38196299b983c55b8 (DEFAULT_TABLE table=test-table [id=6a3c7b81c9b64105b76595dd9f16f049]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.920810 31496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baab5eb0b82f45f38196299b983c55b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.920812 31294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baab5eb0b82f45f38196299b983c55b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.921741 31844 tablet_bootstrap.cc:492] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86: Bootstrap starting.
I20260808 02:13:25.922160 31844 tablet_bootstrap.cc:654] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.922246 31704 tablet_service.cc:1467] Processing CreateTablet for tablet baab5eb0b82f45f38196299b983c55b8 (DEFAULT_TABLE table=test-table [id=6a3c7b81c9b64105b76595dd9f16f049]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.922382 31704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baab5eb0b82f45f38196299b983c55b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.922812 31844 tablet_bootstrap.cc:492] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86: No bootstrap required, opened a new log
I20260808 02:13:25.922856 31844 ts_tablet_manager.cc:1397] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:25.923007 31844 raft_consensus.cc:348] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.923062 31844 raft_consensus.cc:374] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.923081 31844 raft_consensus.cc:729] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78651ec895b545b8b150080295c3dc86, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.923123 31844 consensus_queue.cc:260] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.923202 31844 ts_tablet_manager.cc:1428] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.923769 31849 tablet_bootstrap.cc:492] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f: Bootstrap starting.
I20260808 02:13:25.923777 31843 tablet_bootstrap.cc:492] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324: Bootstrap starting.
I20260808 02:13:25.924124 31843 tablet_bootstrap.cc:654] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.924134 31849 tablet_bootstrap.cc:654] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.924659 31849 tablet_bootstrap.cc:492] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f: No bootstrap required, opened a new log
I20260808 02:13:25.924714 31849 ts_tablet_manager.cc:1397] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:25.924821 31849 raft_consensus.cc:348] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.924903 31849 raft_consensus.cc:374] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.924932 31849 raft_consensus.cc:729] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003b2fd3915a4bd29fdeb293c2a6352f, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.924979 31849 consensus_queue.cc:260] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.925081 31849 ts_tablet_manager.cc:1428] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.925415 31843 tablet_bootstrap.cc:492] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324: No bootstrap required, opened a new log
I20260808 02:13:25.925467 31843 ts_tablet_manager.cc:1397] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:25.925631 31843 raft_consensus.cc:348] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.925701 31843 raft_consensus.cc:374] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.925726 31843 raft_consensus.cc:729] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54a68c96f614a5abc04bc297850f324, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.925762 31843 consensus_queue.cc:260] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.925848 31843 ts_tablet_manager.cc:1428] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:25.930363 31908 raft_consensus.cc:482] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:25.930420 31908 raft_consensus.cc:504] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.930538 31908 leader_election.cc:288] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459), b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053)
I20260808 02:13:25.930725 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baab5eb0b82f45f38196299b983c55b8" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54a68c96f614a5abc04bc297850f324" is_pre_election: true
I20260808 02:13:25.930727 31326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baab5eb0b82f45f38196299b983c55b8" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" is_pre_election: true
I20260808 02:13:25.930794 31733 raft_consensus.cc:2454] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 0.
I20260808 02:13:25.930806 31326 raft_consensus.cc:2454] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 0.
I20260808 02:13:25.930953 31431 leader_election.cc:302] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003b2fd3915a4bd29fdeb293c2a6352f, 78651ec895b545b8b150080295c3dc86; no voters:
I20260808 02:13:25.931051 31908 raft_consensus.cc:2781] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:25.931105 31908 raft_consensus.cc:482] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:25.931133 31908 raft_consensus.cc:3037] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.931998 31908 raft_consensus.cc:504] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.932124 31908 leader_election.cc:288] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 election: Requested vote from peers 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459), b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053)
I20260808 02:13:25.932289 31326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baab5eb0b82f45f38196299b983c55b8" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003b2fd3915a4bd29fdeb293c2a6352f"
I20260808 02:13:25.932350 31326 raft_consensus.cc:3037] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.933071 31326 raft_consensus.cc:2454] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 1.
I20260808 02:13:25.933166 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baab5eb0b82f45f38196299b983c55b8" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54a68c96f614a5abc04bc297850f324"
I20260808 02:13:25.933221 31733 raft_consensus.cc:3037] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.933239 31431 leader_election.cc:302] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003b2fd3915a4bd29fdeb293c2a6352f, 78651ec895b545b8b150080295c3dc86; no voters:
I20260808 02:13:25.933319 31908 raft_consensus.cc:2781] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:25.933355 31908 raft_consensus.cc:686] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 1 LEADER]: Becoming Leader. State: Replica: 78651ec895b545b8b150080295c3dc86, State: Running, Role: LEADER
I20260808 02:13:25.933393 31908 consensus_queue.cc:237] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.933887 31733 raft_consensus.cc:2454] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 1.
I20260808 02:13:25.934662 31094 catalog_manager.cc:5495] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78651ec895b545b8b150080295c3dc86 (127.19.58.194). New cstate: current_term: 1 leader_uuid: "78651ec895b545b8b150080295c3dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:25.943115 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.940832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43622 (local address 127.19.58.194:44699)
0808 02:13:25.940873 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.940874 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.940900 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:13:25.940970 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.940971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.940979 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.941019 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:25.941126 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.941576 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.942598 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.942768 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.942809 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.942954 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:25.942977 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:25.943007 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.943058 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.943071 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:13:25.943897 31733 raft_consensus.cc:1261] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Refusing update from remote peer 78651ec895b545b8b150080295c3dc86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:25.943908 31326 raft_consensus.cc:1261] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Refusing update from remote peer 78651ec895b545b8b150080295c3dc86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:25.944097 31894 consensus_queue.cc:1035] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:25.944213 31908 consensus_queue.cc:1035] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:25.947438 31092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:25.947516 31092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:25.948122 31092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72: Sending DeleteTablet for 3 replicas of tablet baab5eb0b82f45f38196299b983c55b8
I20260808 02:13:25.948410 31704 tablet_service.cc:1514] Processing DeleteTablet for tablet baab5eb0b82f45f38196299b983c55b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:45092
I20260808 02:13:25.948390 31496 tablet_service.cc:1514] Processing DeleteTablet for tablet baab5eb0b82f45f38196299b983c55b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:43608
I20260808 02:13:25.948486 31873 tablet_replica.cc:330] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324: stopping tablet replica
I20260808 02:13:25.948529 31873 raft_consensus.cc:2229] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:25.948532 31875 tablet_replica.cc:330] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86: stopping tablet replica
I20260808 02:13:25.948565 31873 raft_consensus.cc:2258] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.948573 31875 raft_consensus.cc:2229] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:25.948647 31875 raft_consensus.cc:2258] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.948580 31092 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:25.948712 31092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:25.948912 31294 tablet_service.cc:1514] Processing DeleteTablet for tablet baab5eb0b82f45f38196299b983c55b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:41442
I20260808 02:13:25.948990 31872 tablet_replica.cc:330] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f: stopping tablet replica
I20260808 02:13:25.948998 31873 ts_tablet_manager.cc:1905] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.949031 31872 raft_consensus.cc:2229] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:25.949074 31872 raft_consensus.cc:2258] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.949338 31875 ts_tablet_manager.cc:1905] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.949939 31496 tablet_service.cc:1467] Processing CreateTablet for tablet cf8dfac099204bcaaec346447cc24b96 (DEFAULT_TABLE table=test-table [id=961c305710284228a9e045277d857194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.950110 31496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf8dfac099204bcaaec346447cc24b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.950573 31704 tablet_service.cc:1467] Processing CreateTablet for tablet cf8dfac099204bcaaec346447cc24b96 (DEFAULT_TABLE table=test-table [id=961c305710284228a9e045277d857194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.950753 31873 ts_tablet_manager.cc:1918] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:25.950801 31873 log.cc:1196] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/baab5eb0b82f45f38196299b983c55b8
I20260808 02:13:25.950918 31875 ts_tablet_manager.cc:1918] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:25.950961 31875 log.cc:1196] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/baab5eb0b82f45f38196299b983c55b8
I20260808 02:13:25.951000 31873 ts_tablet_manager.cc:1939] T baab5eb0b82f45f38196299b983c55b8 P b54a68c96f614a5abc04bc297850f324: Deleting consensus metadata
I20260808 02:13:25.951118 31704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf8dfac099204bcaaec346447cc24b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.951135 31875 ts_tablet_manager.cc:1939] T baab5eb0b82f45f38196299b983c55b8 P 78651ec895b545b8b150080295c3dc86: Deleting consensus metadata
I20260808 02:13:25.951313 31844 tablet_bootstrap.cc:492] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86: Bootstrap starting.
I20260808 02:13:25.951489 31294 tablet_service.cc:1467] Processing CreateTablet for tablet cf8dfac099204bcaaec346447cc24b96 (DEFAULT_TABLE table=test-table [id=961c305710284228a9e045277d857194]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:25.951592 31294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf8dfac099204bcaaec346447cc24b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:25.951690 31844 tablet_bootstrap.cc:654] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.952225 31843 tablet_bootstrap.cc:492] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324: Bootstrap starting.
I20260808 02:13:25.952214 31075 catalog_manager.cc:4841] TS 78651ec895b545b8b150080295c3dc86 (127.19.58.194:44699): tablet baab5eb0b82f45f38196299b983c55b8 (table test-table [id=6a3c7b81c9b64105b76595dd9f16f049]) successfully deleted
I20260808 02:13:25.952720 31843 tablet_bootstrap.cc:654] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.953027 31849 tablet_bootstrap.cc:492] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f: Bootstrap starting.
I20260808 02:13:25.953300 31843 tablet_bootstrap.cc:492] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324: No bootstrap required, opened a new log
I20260808 02:13:25.953320 31849 tablet_bootstrap.cc:654] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:25.953346 31843 ts_tablet_manager.cc:1397] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:25.953490 31843 raft_consensus.cc:348] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.953560 31843 raft_consensus.cc:374] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.953585 31843 raft_consensus.cc:729] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b54a68c96f614a5abc04bc297850f324, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.953621 31843 consensus_queue.cc:260] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.953706 31843 ts_tablet_manager.cc:1428] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:25.953986 31849 tablet_bootstrap.cc:492] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f: No bootstrap required, opened a new log
I20260808 02:13:25.954036 31849 ts_tablet_manager.cc:1397] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:13:25.954058 31070 catalog_manager.cc:4841] TS b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053): tablet baab5eb0b82f45f38196299b983c55b8 (table test-table [id=6a3c7b81c9b64105b76595dd9f16f049]) successfully deleted
I20260808 02:13:25.954126 31849 raft_consensus.cc:348] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.954197 31849 raft_consensus.cc:374] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.954221 31849 raft_consensus.cc:729] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003b2fd3915a4bd29fdeb293c2a6352f, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.954257 31849 consensus_queue.cc:260] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.954339 31849 ts_tablet_manager.cc:1428] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.954418 31872 ts_tablet_manager.cc:1905] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.954787 31844 tablet_bootstrap.cc:492] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86: No bootstrap required, opened a new log
I20260808 02:13:25.954826 31844 ts_tablet_manager.cc:1397] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 02:13:25.954952 31844 raft_consensus.cc:348] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.955009 31844 raft_consensus.cc:374] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:25.955027 31844 raft_consensus.cc:729] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78651ec895b545b8b150080295c3dc86, State: Initialized, Role: FOLLOWER
I20260808 02:13:25.955070 31844 consensus_queue.cc:260] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.955152 31844 ts_tablet_manager.cc:1428] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:25.955799 31872 ts_tablet_manager.cc:1918] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:25.955861 31872 log.cc:1196] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/baab5eb0b82f45f38196299b983c55b8
I20260808 02:13:25.956043 31872 ts_tablet_manager.cc:1939] T baab5eb0b82f45f38196299b983c55b8 P 003b2fd3915a4bd29fdeb293c2a6352f: Deleting consensus metadata
I20260808 02:13:25.956332 31070 catalog_manager.cc:4841] TS 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459): tablet baab5eb0b82f45f38196299b983c55b8 (table test-table [id=6a3c7b81c9b64105b76595dd9f16f049]) successfully deleted
I20260808 02:13:25.963517 31908 raft_consensus.cc:482] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:25.963567 31908 raft_consensus.cc:504] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.963653 31908 leader_election.cc:288] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459), b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053)
I20260808 02:13:25.963842 31326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf8dfac099204bcaaec346447cc24b96" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" is_pre_election: true
I20260808 02:13:25.963924 31326 raft_consensus.cc:2454] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 0.
I20260808 02:13:25.963883 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf8dfac099204bcaaec346447cc24b96" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54a68c96f614a5abc04bc297850f324" is_pre_election: true
I20260808 02:13:25.963965 31733 raft_consensus.cc:2454] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 0.
I20260808 02:13:25.964066 31431 leader_election.cc:302] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003b2fd3915a4bd29fdeb293c2a6352f, 78651ec895b545b8b150080295c3dc86; no voters:
I20260808 02:13:25.964149 31908 raft_consensus.cc:2781] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:25.964201 31908 raft_consensus.cc:482] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:25.964227 31908 raft_consensus.cc:3037] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.964921 31908 raft_consensus.cc:504] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.965041 31908 leader_election.cc:288] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 election: Requested vote from peers 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459), b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053)
I20260808 02:13:25.965183 31326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf8dfac099204bcaaec346447cc24b96" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "003b2fd3915a4bd29fdeb293c2a6352f"
I20260808 02:13:25.965246 31326 raft_consensus.cc:3037] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.965821 31326 raft_consensus.cc:2454] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 1.
I20260808 02:13:25.965922 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf8dfac099204bcaaec346447cc24b96" candidate_uuid: "78651ec895b545b8b150080295c3dc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b54a68c96f614a5abc04bc297850f324"
I20260808 02:13:25.965965 31431 leader_election.cc:302] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 003b2fd3915a4bd29fdeb293c2a6352f, 78651ec895b545b8b150080295c3dc86; no voters:
I20260808 02:13:25.965991 31733 raft_consensus.cc:3037] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:25.966033 31908 raft_consensus.cc:2781] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:25.966073 31908 raft_consensus.cc:686] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 1 LEADER]: Becoming Leader. State: Replica: 78651ec895b545b8b150080295c3dc86, State: Running, Role: LEADER
I20260808 02:13:25.966114 31908 consensus_queue.cc:237] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } }
I20260808 02:13:25.966549 31092 catalog_manager.cc:5495] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78651ec895b545b8b150080295c3dc86 (127.19.58.194). New cstate: current_term: 1 leader_uuid: "78651ec895b545b8b150080295c3dc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78651ec895b545b8b150080295c3dc86" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:25.966641 31733 raft_consensus.cc:2454] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78651ec895b545b8b150080295c3dc86 in term 1.
I20260808 02:13:25.976470 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.974377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:57597 (local address 127.19.58.194:44699)
0808 02:13:25.974414 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:25.974416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:25.974477 (+ 61us) server_negotiation.cc:407] Connection header received
0808 02:13:25.974535 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:25.974536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:25.974544 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:25.974594 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:25.974697 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.975160 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.975927 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.976107 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.976159 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.976265 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:25.976296 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:25.976334 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:25.976381 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 02:13:25.976398 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":16}
I20260808 02:13:25.976521 31927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:25.974332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44699 (local address 127.19.58.194:57597)
0808 02:13:25.974435 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:25.974445 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:25.974493 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:25.974597 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:25.974598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:25.974599 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 02:13:25.974646 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.974647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.975223 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.975223 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.975858 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:25.975860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:25.976160 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:25.976161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:25.976209 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:25.976216 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 02:13:25.976339 (+ 123us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 02:13:25.976341 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:13:25.976380 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 02:13:25.976426 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":3}
I20260808 02:13:25.978294 31733 raft_consensus.cc:1261] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Refusing update from remote peer 78651ec895b545b8b150080295c3dc86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:25.978314 31326 raft_consensus.cc:1261] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Refusing update from remote peer 78651ec895b545b8b150080295c3dc86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:25.978484 31915 consensus_queue.cc:1035] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "003b2fd3915a4bd29fdeb293c2a6352f" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 39459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:25.978609 31894 consensus_queue.cc:1035] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b54a68c96f614a5abc04bc297850f324" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:25.988372 31094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:25.988452 31094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:25.989017 31094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37495041c5e24e2ca33ff4fb0a218f72: Sending DeleteTablet for 3 replicas of tablet cf8dfac099204bcaaec346447cc24b96
I20260808 02:13:25.989217 31294 tablet_service.cc:1514] Processing DeleteTablet for tablet cf8dfac099204bcaaec346447cc24b96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:41442
I20260808 02:13:25.989240 31704 tablet_service.cc:1514] Processing DeleteTablet for tablet cf8dfac099204bcaaec346447cc24b96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:45092
I20260808 02:13:25.989319 31873 tablet_replica.cc:330] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324: stopping tablet replica
I20260808 02:13:25.989323 31872 tablet_replica.cc:330] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f: stopping tablet replica
I20260808 02:13:25.989368 31873 raft_consensus.cc:2229] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:25.989372 31872 raft_consensus.cc:2229] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:25.989413 31872 raft_consensus.cc:2258] T cf8dfac099204bcaaec346447cc24b96 P 003b2fd3915a4bd29fdeb293c2a6352f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.989413 31873 raft_consensus.cc:2258] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.989563 31496 tablet_service.cc:1514] Processing DeleteTablet for tablet cf8dfac099204bcaaec346447cc24b96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:25 UTC) from {username='oryx', principal='oryx/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:43608
I20260808 02:13:25.989651 31875 tablet_replica.cc:330] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86: stopping tablet replica
I20260808 02:13:25.989701 31875 raft_consensus.cc:2229] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:25.989750 31875 raft_consensus.cc:2258] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:25.989794 31873 ts_tablet_manager.cc:1905] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.989988 31875 ts_tablet_manager.cc:1905] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:25.991286 31875 ts_tablet_manager.cc:1918] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:25.991343 31875 log.cc:1196] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/cf8dfac099204bcaaec346447cc24b96
I20260808 02:13:25.991333 31873 ts_tablet_manager.cc:1918] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 02:13:25.991381 31873 log.cc:1196] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/cf8dfac099204bcaaec346447cc24b96
I20260808 02:13:25.991536 31875 ts_tablet_manager.cc:1939] T cf8dfac099204bcaaec346447cc24b96 P 78651ec895b545b8b150080295c3dc86: Deleting consensus metadata
I20260808 02:13:25.991595 31873 ts_tablet_manager.cc:1939] T cf8dfac099204bcaaec346447cc24b96 P b54a68c96f614a5abc04bc297850f324: Deleting consensus metadata
I20260808 02:13:25.991804 31075 catalog_manager.cc:4841] TS 78651ec895b545b8b150080295c3dc86 (127.19.58.194:44699): tablet cf8dfac099204bcaaec346447cc24b96 (table test-table [id=961c305710284228a9e045277d857194]) successfully deleted
I20260808 02:13:25.991892 31070 catalog_manager.cc:4841] TS b54a68c96f614a5abc04bc297850f324 (127.19.58.195:35053): tablet cf8dfac099204bcaaec346447cc24b96 (table test-table [id=961c305710284228a9e045277d857194]) successfully deleted
I20260808 02:13:25.992184 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 31209
W20260808 02:13:26.001876 31070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:39459 (error 108)
I20260808 02:13:26.002563 31973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:26.002368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:39459 (local address 127.0.0.1:41458)
0808 02:13:26.002473 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:26.002520 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:39459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":55,"threads_started":1}
I20260808 02:13:26.002614 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 31410
W20260808 02:13:26.007270 31070 catalog_manager.cc:4584] TS 003b2fd3915a4bd29fdeb293c2a6352f (127.19.58.193:39459): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf8dfac099204bcaaec346447cc24b96: Network error: Client connection negotiation failed: client connection to 127.19.58.193:39459: connect: Connection refused (error 111)
I20260808 02:13:26.014309 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 31616
I20260808 02:13:26.023933 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 31049
2026-08-08T02:13:26Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2656 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:26 dist-test-slave-1l3l krb5kdc[31979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:26 dist-test-slave-1l3l krb5kdc[31979](info): set up 2 sockets
Aug 08 02:13:26 dist-test-slave-1l3l krb5kdc[31979](info): commencing operation
krb5kdc: starting...
W20260808 02:13:27.145296 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:27 dist-test-slave-1l3l krb5kdc[31979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:13:27.260535 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:44763
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:44763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:27.339815 31998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:27.340021 31998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:27.340051 31998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:27.341158 31998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:27.341197 31998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:27.341214 31998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:27.341229 31998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:27.341245 31998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:27.342670 31998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:44763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:44763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:27.342908 31998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:27.343112 31998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:27.347661 32007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:27.347681 32010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:27.347733 32008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:27.347864 31998 server_base.cc:1034] running on GCE node
I20260808 02:13:27.348120 31998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:27.348385 31998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:27.349524 31998 hybrid_clock.cc:648] HybridClock initialized: now 1786155207349505 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T02:13:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1333 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:27 dist-test-slave-1l3l krb5kdc[32016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:27 dist-test-slave-1l3l krb5kdc[32016](info): set up 2 sockets
Aug 08 02:13:27 dist-test-slave-1l3l krb5kdc[32016](info): commencing operation
krb5kdc: starting...
W20260808 02:13:28.471035 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:28 dist-test-slave-1l3l krb5kdc[32016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:13:28.582437 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42055
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:42055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:28.661736 32039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:28.661935 32039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:28.661968 32039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:28.663076 32039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:28.663117 32039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:28.663136 32039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:28.663152 32039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:28.663168 32039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:28.664646 32039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:42055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:28.664870 32039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:28.665077 32039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:28.669662 32046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:28.669697 32045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:28.669690 32048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:28.669727 32039 server_base.cc:1034] running on GCE node
I20260808 02:13:28.670051 32039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:28.670285 32039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:28.671420 32039 hybrid_clock.cc:648] HybridClock initialized: now 1786155208671407 us; error 31 us; skew 500 ppm
Aug 08 02:13:28 dist-test-slave-1l3l krb5kdc[32016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:28.678404 32039 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:13:28.678805 32039 webserver.cc:460] Webserver started at http://127.19.58.254:43979/ using document root <none> and password file <none>
I20260808 02:13:28.679024 32039 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:28.679076 32039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:28.679201 32039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:28.680258 32039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "c537633933274ed3a509e9fc234f7239"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "39eb81801de44c3cb55c4032145a84ec"
server_key_iv: "a462a4d0e6f9c0903886159b01ec8493"
server_key_version: "encryptionkey@0"
I20260808 02:13:28.680583 32039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "c537633933274ed3a509e9fc234f7239"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "39eb81801de44c3cb55c4032145a84ec"
server_key_iv: "a462a4d0e6f9c0903886159b01ec8493"
server_key_version: "encryptionkey@0"
I20260808 02:13:28.681789 32039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:28.682468 32060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:28.682646 32039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:28.682734 32039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "c537633933274ed3a509e9fc234f7239"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "39eb81801de44c3cb55c4032145a84ec"
server_key_iv: "a462a4d0e6f9c0903886159b01ec8493"
server_key_version: "encryptionkey@0"
I20260808 02:13:28.682793 32039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:28.694155 32039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:28.695854 32039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:28.695991 32039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:28.699648 32039 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:42055
I20260808 02:13:28.699683 32137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:42055 every 8 connection(s)
I20260808 02:13:28.700057 32039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:28.700882 32138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:28.703056 32138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239: Bootstrap starting.
I20260808 02:13:28.703601 32138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:28.703910 32138 log.cc:824] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:28.704602 32138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239: No bootstrap required, opened a new log
I20260808 02:13:28.706146 32138 raft_consensus.cc:348] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } }
I20260808 02:13:28.706277 32138 raft_consensus.cc:374] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:28.706312 32138 raft_consensus.cc:729] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c537633933274ed3a509e9fc234f7239, State: Initialized, Role: FOLLOWER
I20260808 02:13:28.706452 32138 consensus_queue.cc:260] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } }
I20260808 02:13:28.706512 32138 raft_consensus.cc:388] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:28.706533 32138 raft_consensus.cc:482] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:28.706532 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 32039
I20260808 02:13:28.706579 32138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:28.706636 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:28.706794 19691 external_mini_cluster.cc:1428] Setting key 13c1abaa37ce66169f766a183e70aec6
I20260808 02:13:28.707181 32138 raft_consensus.cc:504] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } }
I20260808 02:13:28.707293 32138 leader_election.cc:302] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c537633933274ed3a509e9fc234f7239; no voters:
I20260808 02:13:28.707527 32138 leader_election.cc:288] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:28.707563 32146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:28.707758 32146 raft_consensus.cc:686] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 1 LEADER]: Becoming Leader. State: Replica: c537633933274ed3a509e9fc234f7239, State: Running, Role: LEADER
I20260808 02:13:28.707803 32138 sys_catalog.cc:564] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:28.707932 32146 consensus_queue.cc:237] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } }
I20260808 02:13:28.708635 32147 sys_catalog.cc:455] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c537633933274ed3a509e9fc234f7239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } } }
I20260808 02:13:28.708648 32149 sys_catalog.cc:455] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c537633933274ed3a509e9fc234f7239. Latest consensus state: current_term: 1 leader_uuid: "c537633933274ed3a509e9fc234f7239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } } }
I20260808 02:13:28.708772 32147 sys_catalog.cc:458] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:28.708802 32149 sys_catalog.cc:458] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:28.709026 32161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:28.709591 32161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:28.711059 32161 catalog_manager.cc:1347] Generated new cluster ID: e4ef866b98f04d56a03b871368925853
I20260808 02:13:28.711113 32161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:13:28 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:28.718058 32145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:28.707239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.19.58.254:42055)
0808 02:13:28.707404 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:28.707414 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:28.707445 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:13:28.707609 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:28.707616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:28.707725 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:28.707905 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:28.708686 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.710075 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.711005 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.711591 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.711688 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:28.714437 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:28.714441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:28.714442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:28.714453 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:28.716423 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:28.716772 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:28.716775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:28.716775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:28.716891 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:28.717063 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:28.717064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:28.717065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:28.717355 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:28.717534 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:28.717600 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 02:13:28.717846 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260808 02:13:28.723206 32161 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:28.723670 32161 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:28.727057 32161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239: Generated new TSK 0
I20260808 02:13:28.727207 32161 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:13:28.756438 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42055
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:28.835749 32178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:28.835939 32178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:28.835970 32178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:28.837090 32178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:28.837128 32178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:28.837162 32178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:28.838876 32178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:28.839111 32178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:28.839311 32178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:13:28.843972 32178 server_base.cc:1034] running on GCE node
W20260808 02:13:28.843946 32190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:28.843990 32187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:28.843948 32185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:28.844434 32178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:28.844662 32178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:28.845809 32178 hybrid_clock.cc:648] HybridClock initialized: now 1786155208845793 us; error 28 us; skew 500 ppm
Aug 08 02:13:28 dist-test-slave-1l3l krb5kdc[32016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:28.848155 32178 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:13:28.848546 32178 webserver.cc:460] Webserver started at http://127.19.58.193:40947/ using document root <none> and password file <none>
I20260808 02:13:28.848752 32178 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:28.848798 32178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:28.848933 32178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:28.849864 32178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "6da4f98f4264479fb2cc946b155b3fee"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "88ecfca48984b622d7bd60bc30fd0be2"
server_key_iv: "ecb1b2f31968cd027290de444306c1d1"
server_key_version: "encryptionkey@0"
I20260808 02:13:28.850178 32178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "6da4f98f4264479fb2cc946b155b3fee"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "88ecfca48984b622d7bd60bc30fd0be2"
server_key_iv: "ecb1b2f31968cd027290de444306c1d1"
server_key_version: "encryptionkey@0"
I20260808 02:13:28.851413 32178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 02:13:28.852058 32197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:28.852198 32178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:28.852276 32178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "6da4f98f4264479fb2cc946b155b3fee"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "88ecfca48984b622d7bd60bc30fd0be2"
server_key_iv: "ecb1b2f31968cd027290de444306c1d1"
server_key_version: "encryptionkey@0"
I20260808 02:13:28.852332 32178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:28.866773 32178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:28.868413 32178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:28.868539 32178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:28.868727 32178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:28.868997 32178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:28.869031 32178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:28.869058 32178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:28.869082 32178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:28.874402 32178 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:40869
I20260808 02:13:28.874442 32358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:40869 every 8 connection(s)
I20260808 02:13:28.874719 32178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
Aug 08 02:13:28 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:28.881172 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 32178
I20260808 02:13:28.881269 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:28.881449 19691 external_mini_cluster.cc:1428] Setting key a2c6d68ea3ae9c08fd974a961ad721c8
I20260808 02:13:28.883126 32145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:28.875319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:41321 (local address 127.19.58.254:42055)
0808 02:13:28.875376 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:28.875378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:28.875560 (+ 182us) server_negotiation.cc:407] Connection header received
0808 02:13:28.876047 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:28.876048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:28.876057 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:28.876107 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:28.876926 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.877488 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.878396 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.878591 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.878638 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:28.880564 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:28.880565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:28.880566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:28.880571 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:28.881867 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:28.882272 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:28.882273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:28.882273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:28.882324 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:28.882587 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:28.882587 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:28.882587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:28.882679 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:28.882723 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:28.882999 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 02:13:28.883064 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 02:13:28.883369 32361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:28.875286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:41321)
0808 02:13:28.875463 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:28.875502 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:28.875971 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:28.876157 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:28.876160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:28.876682 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:28.876857 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:28.876861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.877561 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.877561 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:28.878327 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:28.878329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:28.878644 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:28.878645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:28.878734 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:28.879156 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:28.879164 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:28.880478 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:28.881929 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:28.881936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:28.881937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:28.882207 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:28.882406 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:28.882407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:28.882407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:28.882532 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:28.882727 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:28.882733 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:28.882878 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 02:13:28.882977 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 02:13:28.883120 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260808 02:13:28.883916 32359 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42055
I20260808 02:13:28.884013 32359 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:28.884169 32359 heartbeater.cc:511] Master 127.19.58.254:42055 requested a full tablet report, sending...
I20260808 02:13:28.884670 32079 ts_manager.cc:194] Registered new tserver with Master: 6da4f98f4264479fb2cc946b155b3fee (127.19.58.193:40869)
I20260808 02:13:28.885418 32079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:41321
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:13:28.904239 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42055
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:28.983865 32366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:28.984066 32366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:28.984089 32366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:28.985329 32366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:28.985378 32366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:28.985415 32366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:28.986888 32366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:28.987082 32366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:28.987252 32366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:28.991972 32374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:28.991973 32372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:28.991987 32366 server_base.cc:1034] running on GCE node
W20260808 02:13:28.992040 32377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:28.992400 32366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:28.992625 32366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:28.993775 32366 hybrid_clock.cc:648] HybridClock initialized: now 1786155208993762 us; error 30 us; skew 500 ppm
Aug 08 02:13:28 dist-test-slave-1l3l krb5kdc[32016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:28.996119 32366 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:13:28.996523 32366 webserver.cc:460] Webserver started at http://127.19.58.194:33579/ using document root <none> and password file <none>
I20260808 02:13:28.996716 32366 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:28.996762 32366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:28.996897 32366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:28.997885 32366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "ab5f5c7c1e8c46c29fcfaad52cdaeeac"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "779d3b282a861ed3357b178b849a6af4"
server_key_iv: "566d50e13b9280633e8efdf8652f09fd"
server_key_version: "encryptionkey@0"
I20260808 02:13:28.998242 32366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "ab5f5c7c1e8c46c29fcfaad52cdaeeac"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "779d3b282a861ed3357b178b849a6af4"
server_key_iv: "566d50e13b9280633e8efdf8652f09fd"
server_key_version: "encryptionkey@0"
I20260808 02:13:28.999433 32366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 02:13:29.000147 32387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.000293 32366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.000371 32366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "ab5f5c7c1e8c46c29fcfaad52cdaeeac"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "779d3b282a861ed3357b178b849a6af4"
server_key_iv: "566d50e13b9280633e8efdf8652f09fd"
server_key_version: "encryptionkey@0"
I20260808 02:13:29.000437 32366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:29.008180 32366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:29.009980 32366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:29.010114 32366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:29.010357 32366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:29.010702 32366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:29.010744 32366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.010773 32366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:29.010797 32366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.016630 32366 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:39597
I20260808 02:13:29.016666 32557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:39597 every 8 connection(s)
I20260808 02:13:29.016960 32366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:29.018426 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 32366
I20260808 02:13:29.018545 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:29.018764 19691 external_mini_cluster.cc:1428] Setting key 5db7110200ac34f91f513da1aeb040de
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155208, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:29.027405 32145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.017597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:59379 (local address 127.19.58.254:42055)
0808 02:13:29.017650 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.017652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.017818 (+ 166us) server_negotiation.cc:407] Connection header received
0808 02:13:29.018272 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.018273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.018282 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.018335 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.019410 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.019945 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.021259 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.021544 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.021614 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.024026 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:29.024027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:29.024027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:29.024035 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:29.025740 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:29.026246 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:29.026248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:29.026248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:29.026310 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:29.026585 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:29.026586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:29.026586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:29.026739 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:29.026800 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:29.027136 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 02:13:29.027284 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:13:29.027503 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.017562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:59379)
0808 02:13:29.017704 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.017743 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.018191 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.018410 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.018413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.019128 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.019332 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.019338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.020040 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.020041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.021167 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.021171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.021634 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.021635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.021745 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.022234 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.022241 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.023918 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:29.025822 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:29.025829 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:29.025831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:29.026134 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:29.026371 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:29.026372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:29.026372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:29.026520 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:29.026813 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:29.026821 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:29.026988 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 02:13:29.027094 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 02:13:29.027269 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
I20260808 02:13:29.029172 32559 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42055
I20260808 02:13:29.029277 32559 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:29.029618 32559 heartbeater.cc:511] Master 127.19.58.254:42055 requested a full tablet report, sending...
I20260808 02:13:29.030494 32079 ts_manager.cc:194] Registered new tserver with Master: ab5f5c7c1e8c46c29fcfaad52cdaeeac (127.19.58.194:39597)
I20260808 02:13:29.031042 32079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:59379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:13:29.044394 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42055
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:29.123101 32567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:29.123284 32567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:29.123306 32567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:29.124460 32567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:29.124501 32567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:29.124529 32567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:29.125959 32567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:29.126171 32567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:29.126367 32567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:29.131201 32575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:29.131224 32574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:29.131246 32580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:29.131366 32567 server_base.cc:1034] running on GCE node
I20260808 02:13:29.131645 32567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:29.131879 32567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:29.133014 32567 hybrid_clock.cc:648] HybridClock initialized: now 1786155209133002 us; error 30 us; skew 500 ppm
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155209, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:29.135428 32567 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:13:29.135867 32567 webserver.cc:460] Webserver started at http://127.19.58.195:39467/ using document root <none> and password file <none>
I20260808 02:13:29.136062 32567 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:29.136101 32567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:29.136189 32567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:29.137120 32567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "8e2e3b7d2dc84cb9886e325b9fac731d"
format_stamp: "Formatted at 2026-08-08 02:13:29 on dist-test-slave-1l3l"
server_key: "d7aa29c6f6362c0613c30a3ffb2f5f9a"
server_key_iv: "a1649d185b32879b942e1e3161b9c470"
server_key_version: "encryptionkey@0"
I20260808 02:13:29.137404 32567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "8e2e3b7d2dc84cb9886e325b9fac731d"
format_stamp: "Formatted at 2026-08-08 02:13:29 on dist-test-slave-1l3l"
server_key: "d7aa29c6f6362c0613c30a3ffb2f5f9a"
server_key_iv: "a1649d185b32879b942e1e3161b9c470"
server_key_version: "encryptionkey@0"
I20260808 02:13:29.138582 32567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:29.139178 32587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.139315 32567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:29.139411 32567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "8e2e3b7d2dc84cb9886e325b9fac731d"
format_stamp: "Formatted at 2026-08-08 02:13:29 on dist-test-slave-1l3l"
server_key: "d7aa29c6f6362c0613c30a3ffb2f5f9a"
server_key_iv: "a1649d185b32879b942e1e3161b9c470"
server_key_version: "encryptionkey@0"
I20260808 02:13:29.139465 32567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:29.148805 32567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:29.150382 32567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:29.150488 32567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:29.150650 32567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:29.150914 32567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:29.150946 32567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.150966 32567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:29.150990 32567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.156489 32567 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:36703
I20260808 02:13:29.156558 32762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:36703 every 8 connection(s)
I20260808 02:13:29.156813 32567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:29.158749 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 32567
I20260808 02:13:29.158862 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:29.159039 19691 external_mini_cluster.cc:1428] Setting key fd8003ecdc1c062c39e92015d10575b0
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155209, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:29.165285 32145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.157484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:58299 (local address 127.19.58.254:42055)
0808 02:13:29.157539 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.157541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.157705 (+ 164us) server_negotiation.cc:407] Connection header received
0808 02:13:29.158144 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.158145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.158153 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.158207 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.159283 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.160033 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.160962 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.161143 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.161185 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.163062 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:29.163063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:29.163063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:29.163070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:29.164306 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:29.164633 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:29.164633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:29.164634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:29.164678 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:29.164846 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:29.164847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:29.164847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:29.164937 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:29.164980 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:29.165169 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 02:13:29.165222 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 02:13:29.165481 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.157439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:58299)
0808 02:13:29.157588 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.157628 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.158072 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.158256 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.158259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.158912 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.159185 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.159192 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.160109 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.160110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.160897 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.160900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.161193 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.161194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.161286 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.161725 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.161732 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.162986 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:29.164364 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:29.164367 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:29.164369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:29.164579 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:29.164721 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:29.164721 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:29.164721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:29.164806 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:29.164976 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:29.164979 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:29.165080 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 02:13:29.165150 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:13:29.165262 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260808 02:13:29.165889 32763 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42055
I20260808 02:13:29.165982 32763 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:29.166136 32763 heartbeater.cc:511] Master 127.19.58.254:42055 requested a full tablet report, sending...
I20260808 02:13:29.166483 32079 ts_manager.cc:194] Registered new tserver with Master: 8e2e3b7d2dc84cb9886e325b9fac731d (127.19.58.195:36703)
I20260808 02:13:29.166818 32079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:58299
I20260808 02:13:29.170625 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab.
Entry for principal oryx/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab.
WARNING: no policy specified for oryx/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab.
Entry for principal oryx/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab.
WARNING: no policy specified for oryx/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab.
Entry for principal oryx/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab.
I20260808 02:13:29.236317 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 32178
I20260808 02:13:29.246757 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 32366
I20260808 02:13:29.256681 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 32567
I20260808 02:13:29.267009 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 32039
I20260808 02:13:29.278416 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42055
--webserver_interface=127.19.58.254
--webserver_port=43979
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:42055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:29.357064 312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:29.357255 312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:29.357285 312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:29.358433 312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:29.358479 312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:29.358492 312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:29.358505 312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:29.358515 312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:29.360010 312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:42055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=43979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:29.360236 312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:29.360445 312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:29.365128 319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:29.365195 312 server_base.cc:1034] running on GCE node
W20260808 02:13:29.365139 320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:29.365151 323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:29.365598 312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:29.365871 312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:29.367018 312 hybrid_clock.cc:648] HybridClock initialized: now 1786155209367007 us; error 25 us; skew 500 ppm
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155209, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:29.369441 312 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:13:29.369817 312 webserver.cc:460] Webserver started at http://127.19.58.254:43979/ using document root <none> and password file <none>
I20260808 02:13:29.370015 312 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:29.370064 312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:29.371354 312 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:29.372009 330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.372177 312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 02:13:29.372260 312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "c537633933274ed3a509e9fc234f7239"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "39eb81801de44c3cb55c4032145a84ec"
server_key_iv: "a462a4d0e6f9c0903886159b01ec8493"
server_key_version: "encryptionkey@0"
I20260808 02:13:29.372611 312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:29.402287 312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:29.403996 312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:29.404110 312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:29.407742 312 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:42055
I20260808 02:13:29.407793 405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:42055 every 8 connection(s)
I20260808 02:13:29.408113 312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:29.409487 406 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:13:29.410097 406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239: Bootstrap starting.
I20260808 02:13:29.413041 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 312
I20260808 02:13:29.413378 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:40869
--local_ip_for_outbound_sockets=127.19.58.193
--tserver_master_addrs=127.19.58.254:42055
--webserver_port=40947
--webserver_interface=127.19.58.193
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.58.193
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:13:29.419411 406 log.cc:824] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:29.420915 406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:13:29.421172 406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239: Bootstrap complete.
I20260808 02:13:29.423265 406 raft_consensus.cc:348] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } }
I20260808 02:13:29.423486 406 raft_consensus.cc:729] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c537633933274ed3a509e9fc234f7239, State: Initialized, Role: FOLLOWER
I20260808 02:13:29.423621 406 consensus_queue.cc:260] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } }
I20260808 02:13:29.423686 406 raft_consensus.cc:388] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:29.423754 406 raft_consensus.cc:482] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:29.423802 406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:13:29.424741 406 raft_consensus.cc:504] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } }
I20260808 02:13:29.424829 406 leader_election.cc:302] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c537633933274ed3a509e9fc234f7239; no voters:
I20260808 02:13:29.424983 406 leader_election.cc:288] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:13:29.425031 412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:13:29.425160 412 raft_consensus.cc:686] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [term 2 LEADER]: Becoming Leader. State: Replica: c537633933274ed3a509e9fc234f7239, State: Running, Role: LEADER
I20260808 02:13:29.425240 406 sys_catalog.cc:564] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:29.425242 412 consensus_queue.cc:237] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } }
I20260808 02:13:29.425534 412 sys_catalog.cc:455] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c537633933274ed3a509e9fc234f7239. Latest consensus state: current_term: 2 leader_uuid: "c537633933274ed3a509e9fc234f7239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } } }
I20260808 02:13:29.425603 412 sys_catalog.cc:458] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:29.425738 414 sys_catalog.cc:455] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c537633933274ed3a509e9fc234f7239" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c537633933274ed3a509e9fc234f7239" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42055 } } }
I20260808 02:13:29.425803 414 sys_catalog.cc:458] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:29.426071 418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:29.426569 418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:29.426731 418 catalog_manager.cc:1259] Loaded cluster ID: e4ef866b98f04d56a03b871368925853
I20260808 02:13:29.426950 418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:29.427793 418 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:29.427958 418 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c537633933274ed3a509e9fc234f7239: Loaded TSK: 0
I20260808 02:13:29.428164 418 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:13:29.516011 409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:29.516202 409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:29.516233 409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:29.517335 409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:29.517376 409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:29.517418 409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:29.518813 409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.193.keytab
--principal=oryx/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:40869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=40947
--tserver_master_addrs=127.19.58.254:42055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:29.519028 409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:29.519244 409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:29.525067 451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:29.525084 449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:29.525318 409 server_base.cc:1034] running on GCE node
W20260808 02:13:29.525067 453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:29.525589 409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:29.525867 409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:29.527017 409 hybrid_clock.cc:648] HybridClock initialized: now 1786155209527004 us; error 33 us; skew 500 ppm
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155209, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:29.529592 409 init.cc:377] Logged in from keytab as oryx/127.19.58.193@KRBTEST.COM (short username oryx)
I20260808 02:13:29.530028 409 webserver.cc:460] Webserver started at http://127.19.58.193:40947/ using document root <none> and password file <none>
I20260808 02:13:29.530232 409 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:29.530292 409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:29.531579 409 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:29.532191 463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.532347 409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.532438 409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "6da4f98f4264479fb2cc946b155b3fee"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "88ecfca48984b622d7bd60bc30fd0be2"
server_key_iv: "ecb1b2f31968cd027290de444306c1d1"
server_key_version: "encryptionkey@0"
I20260808 02:13:29.532768 409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:29.547137 409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:29.549157 409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:29.549269 409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:29.549495 409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:29.549805 409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:29.549845 409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.549875 409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:29.549898 409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.556588 409 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:40869
I20260808 02:13:29.556667 618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:40869 every 8 connection(s)
I20260808 02:13:29.556977 409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:29.559758 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 409
I20260808 02:13:29.560055 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:39597
--local_ip_for_outbound_sockets=127.19.58.194
--tserver_master_addrs=127.19.58.254:42055
--webserver_port=33579
--webserver_interface=127.19.58.194
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.58.194
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.565366 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.557644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:54877)
0808 02:13:29.557834 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.557879 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.558471 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.559042 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.559045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.559604 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.559843 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.559851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.561057 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.561058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.561960 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.561964 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.562529 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.562530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.562650 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.563103 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.563112 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.565178 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
W20260808 02:13:29.565738 621 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:42055 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.58.254:42055: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
W20260808 02:13:29.565760 624 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.557743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:54877 (local address 127.19.58.254:42055)
0808 02:13:29.558666 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.558674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.558709 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:13:29.558821 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.558827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.558876 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.558996 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.559955 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.560959 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.562037 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.562449 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.562545 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.565315 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:54877: BlockingRecv error: recv got EOF from 127.19.58.193:54877 (error 108)
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":78,"threads_started":1}
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.569950 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.565965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:39085)
0808 02:13:29.566117 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.566127 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.566166 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.566318 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.566319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.566470 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.566519 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.566521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.567116 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.567117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.567601 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.567603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.567967 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.567968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.568030 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.568387 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.568391 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.569892 (+ 1501us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135}
W20260808 02:13:29.570044 628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.566017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:39085 (local address 127.19.58.254:42055)
0808 02:13:29.566140 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.566142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.566152 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:13:29.566212 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.566213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.566223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.566586 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.566613 (+ 27us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.567048 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.567669 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.567909 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.567964 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.569987 (+ 2023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:39085: BlockingRecv error: recv got EOF from 127.19.58.193:39085 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.573992 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.570363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:59579)
0808 02:13:29.570444 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.570453 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.570492 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.570598 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.570599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.570754 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.570778 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.570779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.571361 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.571361 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.571865 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.571867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.572176 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.572177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.572224 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.572563 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.572568 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.573935 (+ 1367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 02:13:29.574064 628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.570409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:59579 (local address 127.19.58.254:42055)
0808 02:13:29.570444 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.570446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.570480 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:13:29.570544 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.570544 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.570553 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.570598 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.570851 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.571292 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.571938 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.572123 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.572173 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.574019 (+ 1846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:59579: BlockingRecv error: recv got EOF from 127.19.58.193:59579 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 02:13:29.574287 621 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:13:29.645148 627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:29.645345 627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:29.645368 627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:29.646473 627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:29.646507 627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:29.646531 627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:29.647886 627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.194.keytab
--principal=oryx/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:39597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=33579
--tserver_master_addrs=127.19.58.254:42055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:29.648053 627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:29.648221 627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:29.652838 634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:29.653000 627 server_base.cc:1034] running on GCE node
W20260808 02:13:29.652863 633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:29.652971 637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:29.653301 627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:29.653518 627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:29.654659 627 hybrid_clock.cc:648] HybridClock initialized: now 1786155209654645 us; error 28 us; skew 500 ppm
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155209, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:29.657112 627 init.cc:377] Logged in from keytab as oryx/127.19.58.194@KRBTEST.COM (short username oryx)
I20260808 02:13:29.657554 627 webserver.cc:460] Webserver started at http://127.19.58.194:33579/ using document root <none> and password file <none>
I20260808 02:13:29.657745 627 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:29.657809 627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:29.659116 627 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:29.659639 648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.659806 627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:29.659888 627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "ab5f5c7c1e8c46c29fcfaad52cdaeeac"
format_stamp: "Formatted at 2026-08-08 02:13:28 on dist-test-slave-1l3l"
server_key: "779d3b282a861ed3357b178b849a6af4"
server_key_iv: "566d50e13b9280633e8efdf8652f09fd"
server_key_version: "encryptionkey@0"
I20260808 02:13:29.660187 627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:29.674834 627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:29.676651 627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:29.676759 627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:29.676957 627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:29.677201 627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:29.677230 627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.677249 627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:29.677265 627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.682655 627 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:39597
I20260808 02:13:29.682714 804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:39597 every 8 connection(s)
I20260808 02:13:29.683000 627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:29.685956 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 627
I20260808 02:13:29.686232 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:36703
--local_ip_for_outbound_sockets=127.19.58.195
--tserver_master_addrs=127.19.58.254:42055
--webserver_port=39467
--webserver_interface=127.19.58.195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.58.195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.690330 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.683627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:39295)
0808 02:13:29.683814 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.683853 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.684290 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.684545 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.684548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.685074 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.685264 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.685269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.685960 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.685961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.686866 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.686869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.687294 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.687295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.687435 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.687940 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.687950 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.690161 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":117,"threads_started":1}
W20260808 02:13:29.690340 628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.683660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:39295 (local address 127.19.58.254:42055)
0808 02:13:29.683726 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.683728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.683920 (+ 192us) server_negotiation.cc:407] Connection header received
0808 02:13:29.684429 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.684430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.684441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.684489 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.685335 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.685871 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.686950 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.687207 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.687272 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.690274 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:39295: BlockingRecv error: recv got EOF from 127.19.58.194:39295 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 02:13:29.690572 806 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:42055 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.58.254:42055: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.694883 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.690782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:51813)
0808 02:13:29.690835 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.690847 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.690904 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.691022 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.691023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.691158 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.691209 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.691210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.691808 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.691809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.692562 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.692565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.692875 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.692876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.692938 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.693281 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.693287 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.694835 (+ 1548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 02:13:29.694996 628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.690826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:51813 (local address 127.19.58.254:42055)
0808 02:13:29.690861 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.690863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.690889 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:13:29.690956 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.690957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.690965 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.691007 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.691271 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.691728 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.692638 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.692815 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.692857 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.694938 (+ 2081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:51813: BlockingRecv error: recv got EOF from 127.19.58.194:51813 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.698882 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.695140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:42119)
0808 02:13:29.695171 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.695179 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.695207 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.695354 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.695355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.695457 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.695482 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.695483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.696301 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.696302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.696753 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.696754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.697159 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.697159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.697190 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.697523 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.697527 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.698836 (+ 1309us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 02:13:29.698971 628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.695197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:42119 (local address 127.19.58.254:42055)
0808 02:13:29.695239 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.695241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.695249 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:29.695271 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.695272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.695280 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.695352 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.695544 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.696225 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.696823 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.697096 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.697166 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.698918 (+ 1752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:42119: BlockingRecv error: recv got EOF from 127.19.58.194:42119 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 02:13:29.699019 806 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:13:29.767225 810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:29.767410 810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:29.767431 810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:29.768563 810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:29.768602 810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:29.768626 810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:29.769997 810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/krb5kdc/oryx_127.19.58.195.keytab
--principal=oryx/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:36703
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=39467
--tserver_master_addrs=127.19.58.254:42055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:29.770181 810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:29.770349 810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:29.774998 816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:29.775084 810 server_base.cc:1034] running on GCE node
W20260808 02:13:29.774987 819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:29.774987 817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:29.775373 810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:29.775585 810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:29.776717 810 hybrid_clock.cc:648] HybridClock initialized: now 1786155209776715 us; error 36 us; skew 500 ppm
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155209, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:29.779080 810 init.cc:377] Logged in from keytab as oryx/127.19.58.195@KRBTEST.COM (short username oryx)
I20260808 02:13:29.779500 810 webserver.cc:460] Webserver started at http://127.19.58.195:39467/ using document root <none> and password file <none>
I20260808 02:13:29.779688 810 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:29.779785 810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:29.781037 810 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:29.781566 828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.781697 810 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:29.781778 810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "8e2e3b7d2dc84cb9886e325b9fac731d"
format_stamp: "Formatted at 2026-08-08 02:13:29 on dist-test-slave-1l3l"
server_key: "d7aa29c6f6362c0613c30a3ffb2f5f9a"
server_key_iv: "a1649d185b32879b942e1e3161b9c470"
server_key_version: "encryptionkey@0"
I20260808 02:13:29.782079 810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:29.800091 810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:29.801702 810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:29.801836 810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:29.802034 810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:29.802317 810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:29.802351 810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.802381 810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:29.802409 810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:29.808003 810 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:36703
I20260808 02:13:29.808061 1006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:36703 every 8 connection(s)
I20260808 02:13:29.808358 810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:29.811655 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 810
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.815814 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.809020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:40007)
0808 02:13:29.809169 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.809202 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.809638 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.809816 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.809820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.810329 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.810530 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.810534 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.811131 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.811132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.812235 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.812238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.812624 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.812625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.812779 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.813277 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.813286 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.815612 (+ 2326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
W20260808 02:13:29.815819 628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.809031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:40007 (local address 127.19.58.254:42055)
0808 02:13:29.809077 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.809080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.809275 (+ 195us) server_negotiation.cc:407] Connection header received
0808 02:13:29.809709 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.809710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.809719 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.809763 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.810594 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.811073 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.812310 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.812550 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.812597 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.815733 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:40007: BlockingRecv error: recv got EOF from 127.19.58.195:40007 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 02:13:29.816159 1008 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:42055 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.58.254:42055: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
I20260808 02:13:29.818562 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.812151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.19.58.254:42055)
0808 02:13:29.812188 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.812189 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.812243 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:13:29.812320 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.812321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.812332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.812390 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.813022 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.813545 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.814362 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.814599 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.814650 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.815735 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:29.815739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:29.815740 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:29.815752 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:29.817347 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:29.817664 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:29.817668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:29.817668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:29.817755 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:29.817927 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:29.817928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:29.817928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:29.818137 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:29.818263 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:29.818372 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 02:13:29.818475 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.820188 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.816363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:43579)
0808 02:13:29.816415 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.816423 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.816467 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.816581 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.816582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.816693 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.816731 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.816732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.817279 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.817280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.817753 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.817754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.818064 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.818065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.818130 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.818477 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.818482 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.820147 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 02:13:29.820268 628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.816399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:43579 (local address 127.19.58.254:42055)
0808 02:13:29.816431 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.816433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.816455 (+ 22us) server_negotiation.cc:407] Connection header received
0808 02:13:29.816511 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.816511 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.816517 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.816566 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.816791 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.817217 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.817816 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.818009 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.818059 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.820216 (+ 2157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:43579: BlockingRecv error: recv got EOF from 127.19.58.195:43579 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:29 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:29.823812 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:29.820521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:39159)
0808 02:13:29.820561 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:29.820570 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:29.820603 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:29.820687 (+ 84us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:29.820687 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:29.820781 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:29.820804 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.820805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.821297 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.821298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.821766 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:29.821767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.822043 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.822044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:29.822075 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.822345 (+ 270us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:29.822348 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:29.823754 (+ 1406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 02:13:29.823859 628 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:29.820572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:39159 (local address 127.19.58.254:42055)
0808 02:13:29.820601 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:29.820603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:29.820610 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:29.820633 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:29.820634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:29.820641 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:29.820684 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:29.820846 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.821251 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.821826 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:29.821996 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:29.822040 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:29.823817 (+ 1777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:39159: BlockingRecv error: recv got EOF from 127.19.58.195:39159 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260808 02:13:29.823939 1008 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 02:13:30 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:30 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:30.579062 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:30.574634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:56997)
0808 02:13:30.574770 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:30.574782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:30.574840 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:30.574997 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:30.574998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:30.575219 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:30.575268 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.575271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.575992 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.575993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.576556 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.576559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.576860 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.576861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.576900 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.577328 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:30.577332 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:30.578995 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
W20260808 02:13:30.579164 1014 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:30.574703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:56997 (local address 127.19.58.254:42055)
0808 02:13:30.574814 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:30.574816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:30.574826 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:13:30.574903 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:30.574904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:30.574916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:30.574984 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:30.575337 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.575932 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.576614 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.576805 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.576854 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.579089 (+ 2235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:56997: BlockingRecv error: recv got EOF from 127.19.58.193:56997 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":59,"threads_started":1}
Aug 08 02:13:30 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:30 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:30.703748 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:30.699358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:41853)
0808 02:13:30.699512 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:30.699525 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:30.699584 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:30.699724 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:30.699725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:30.699902 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:30.699953 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.699955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.700615 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.700616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.701156 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.701158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.701504 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.701504 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.701541 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.701952 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:30.701956 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:30.703645 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
W20260808 02:13:30.703835 1014 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:30.699477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:41853 (local address 127.19.58.254:42055)
0808 02:13:30.699543 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:30.699545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:30.699570 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:13:30.699630 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:30.699631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:30.699642 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:30.699689 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:30.700010 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.700550 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.701245 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.701443 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.701502 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.703772 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:41853: BlockingRecv error: recv got EOF from 127.19.58.194:41853 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 02:13:30 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:30 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:30.828707 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:30.824277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:49465)
0808 02:13:30.824412 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:30.824424 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:30.824483 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:30.824593 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:30.824594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:30.824750 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:30.824800 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.824802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.825486 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.825486 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.826070 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:30.826073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.826421 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.826422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:30.826466 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.826869 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:30.826873 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:30.828619 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
W20260808 02:13:30.828770 1014 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:30.824337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:49465 (local address 127.19.58.254:42055)
0808 02:13:30.824397 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:30.824400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:30.824468 (+ 68us) server_negotiation.cc:407] Connection header received
0808 02:13:30.824524 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:30.824524 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:30.824536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:30.824583 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:30.824846 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.825416 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.826144 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:30.826359 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:30.826417 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:30.828717 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:49465: BlockingRecv error: recv got EOF from 127.19.58.195:49465 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 02:13:31 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:31 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:31.584100 1018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:31.579547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:55439)
0808 02:13:31.579675 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:31.579686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:31.579767 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:31.579903 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:31.579903 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:31.580060 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:31.580117 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.580118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.580768 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.580770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.581345 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.581347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.581709 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.581709 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.581785 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.582215 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:31.582219 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:31.584036 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
W20260808 02:13:31.584203 1019 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:31.579615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:55439 (local address 127.19.58.254:42055)
0808 02:13:31.579755 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:31.579758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:31.579766 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:31.579822 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:31.579822 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:31.579832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:31.579892 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:31.580167 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.580687 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.581442 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.581653 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.581704 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.584122 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:55439: BlockingRecv error: recv got EOF from 127.19.58.193:55439 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
Aug 08 02:13:31 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:31 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:31.711447 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:31.704321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:50757)
0808 02:13:31.704558 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:31.704578 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:31.704657 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:31.704895 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:31.704896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:31.705149 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:31.705228 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.705230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.706158 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.706159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.707099 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.707109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.707941 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.707943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.708010 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.708640 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:31.708645 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:31.711347 (+ 2702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":142,"threads_started":1}
W20260808 02:13:31.711596 1019 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:31.704434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:50757 (local address 127.19.58.254:42055)
0808 02:13:31.704532 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:31.704534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:31.704636 (+ 102us) server_negotiation.cc:407] Connection header received
0808 02:13:31.704731 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:31.704732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:31.704757 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:31.704888 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:31.705306 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.706056 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.707252 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.707660 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.707810 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.711476 (+ 3666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:50757: BlockingRecv error: recv got EOF from 127.19.58.194:50757 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 02:13:31 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:31 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:31.835901 1023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:31.829233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:45949)
0808 02:13:31.829442 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:31.829453 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:31.829525 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:31.829699 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:31.829700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:31.829949 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:31.830026 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.830028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.830922 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.830923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.831765 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:31.831767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.832259 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.832260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:31.832333 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.833016 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:31.833020 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:31.835771 (+ 2751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":140,"threads_started":1}
W20260808 02:13:31.836014 1019 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:31.829409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:45949 (local address 127.19.58.254:42055)
0808 02:13:31.829500 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:31.829502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:31.829518 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:13:31.829596 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:31.829597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:31.829609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:31.829691 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:31.830133 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.830836 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.831857 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:31.832166 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:31.832259 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:31.835913 (+ 3654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:45949: BlockingRecv error: recv got EOF from 127.19.58.195:45949 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 02:13:32 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:32 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:32.591277 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:32.584627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:45153)
0808 02:13:32.584858 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:32.584874 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:32.584965 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:32.585153 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:32.585153 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:32.585404 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:32.585507 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.585510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.586533 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.586534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.587300 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.587303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.587848 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.587849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.587951 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.588565 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:32.588569 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:32.591104 (+ 2535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":88,"threads_started":1}
W20260808 02:13:32.591377 1029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:32.584689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:45153 (local address 127.19.58.254:42055)
0808 02:13:32.584861 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:32.584863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:32.584929 (+ 66us) server_negotiation.cc:407] Connection header received
0808 02:13:32.585033 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:32.585034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:32.585058 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:32.585139 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:32.585620 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.586424 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.587411 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.587692 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.587835 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.591272 (+ 3437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:45153: BlockingRecv error: recv got EOF from 127.19.58.193:45153 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
Aug 08 02:13:32 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:32 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:32.718909 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:32.712070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:49341)
0808 02:13:32.712239 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:32.712258 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:32.712371 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:32.712564 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:32.712564 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:32.712922 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:32.713019 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.713022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.714090 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.714091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.714844 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.714846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.715370 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.715370 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.715439 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.716088 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:32.716093 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:32.718727 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
W20260808 02:13:32.718986 1029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:32.712146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:49341 (local address 127.19.58.254:42055)
0808 02:13:32.712244 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:32.712246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:32.712353 (+ 107us) server_negotiation.cc:407] Connection header received
0808 02:13:32.712464 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:32.712465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:32.712485 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:32.712548 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:32.713139 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.714006 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.714946 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.715265 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.715353 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.718894 (+ 3541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:49341: BlockingRecv error: recv got EOF from 127.19.58.194:49341 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 02:13:32 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:32 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:32.843345 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:32.836535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:51023)
0808 02:13:32.836744 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:32.836761 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:32.836851 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:32.837081 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:32.837082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:32.837371 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:32.837442 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.837445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.838458 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.838459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.839242 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:32.839245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.839847 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.839849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:32.839956 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.840580 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:32.840584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:32.843232 (+ 2648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":120,"threads_started":1}
W20260808 02:13:32.843461 1029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:32.836598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:51023 (local address 127.19.58.254:42055)
0808 02:13:32.836673 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:32.836675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:32.836842 (+ 167us) server_negotiation.cc:407] Connection header received
0808 02:13:32.836966 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:32.836967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:32.836991 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:32.837073 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:32.837567 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.838302 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.839352 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:32.839661 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:32.839829 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:32.843385 (+ 3556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:51023: BlockingRecv error: recv got EOF from 127.19.58.195:51023 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 02:13:33 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:33 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:33.598595 1032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:33.591903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:47657)
0808 02:13:33.592101 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:33.592122 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:33.592236 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:33.592447 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:33.592448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:33.592723 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:33.592798 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.592801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.593743 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.593744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.594556 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.594558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.594998 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.594999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.595059 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.595809 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:33.595816 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:33.598496 (+ 2680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
W20260808 02:13:33.598740 1033 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:33.591988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:47657 (local address 127.19.58.254:42055)
0808 02:13:33.592223 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:33.592225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:33.592242 (+ 17us) server_negotiation.cc:407] Connection header received
0808 02:13:33.592326 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:33.592326 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:33.592338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:33.592439 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:33.592894 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.593654 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.594643 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.594935 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.594997 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.598622 (+ 3625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:47657: BlockingRecv error: recv got EOF from 127.19.58.193:47657 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":160,"threads_started":1}
Aug 08 02:13:33 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:33 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:33.729547 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:33.719602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:55679)
0808 02:13:33.719783 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:33.719795 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:33.719880 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:33.720076 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:33.720076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:33.720334 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:33.720417 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.720419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.721350 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.721351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.722407 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.722411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.722937 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.722938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.722998 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.726699 (+ 3701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:33.726705 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:33.729409 (+ 2704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
W20260808 02:13:33.729665 1033 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:33.719680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:55679 (local address 127.19.58.254:42055)
0808 02:13:33.719772 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:33.719774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:33.719863 (+ 89us) server_negotiation.cc:407] Connection header received
0808 02:13:33.719970 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:33.719971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:33.719989 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:33.720062 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:33.720506 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.721261 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.722517 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.722835 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.722921 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.729573 (+ 6652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:55679: BlockingRecv error: recv got EOF from 127.19.58.194:55679 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 02:13:33 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:33 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:33.851186 1035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:33.844143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:37703)
0808 02:13:33.844491 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:33.844510 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:33.844587 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:33.844781 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:33.844782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:33.845101 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:33.845171 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.845173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.846197 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.846198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.847109 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:33.847111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.847583 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.847584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:33.847758 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.848421 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:33.848425 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:33.851106 (+ 2681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":204,"threads_started":1}
W20260808 02:13:33.851325 1033 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:33.844270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:37703 (local address 127.19.58.254:42055)
0808 02:13:33.844360 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:33.844362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:33.844580 (+ 218us) server_negotiation.cc:407] Connection header received
0808 02:13:33.844666 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:33.844667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:33.844694 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:33.844755 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:33.845238 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.846060 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.847210 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:33.847496 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:33.847585 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:33.851222 (+ 3637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:37703: BlockingRecv error: recv got EOF from 127.19.58.195:37703 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 02:13:34 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:34 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:34.603451 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:34.599108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:59407)
0808 02:13:34.599258 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:34.599271 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:34.599342 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:34.599542 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:34.599543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:34.599734 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:34.599793 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.599795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.600414 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.600415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.600960 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.600962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.601247 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.601247 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.601283 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.601708 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:34.601713 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:34.603362 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
W20260808 02:13:34.603549 1039 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:34.599169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:59407 (local address 127.19.58.254:42055)
0808 02:13:34.599432 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:34.599434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:34.599441 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:34.599463 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:34.599463 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:34.599474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:34.599529 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:34.599849 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.600347 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.601019 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.601202 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.601244 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.603480 (+ 2236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:59407: BlockingRecv error: recv got EOF from 127.19.58.193:59407 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":185,"threads_started":1}
Aug 08 02:13:34 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:34 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:34.734642 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:34.730120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:46067)
0808 02:13:34.730279 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:34.730291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:34.730356 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:34.730497 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:34.730498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:34.730676 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:34.730724 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.730727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.731377 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.731378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.731927 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.731929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.732270 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.732271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.732325 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.732855 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:34.732859 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:34.734581 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
W20260808 02:13:34.734745 1039 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:34.730241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:46067 (local address 127.19.58.254:42055)
0808 02:13:34.730291 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:34.730293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:34.730338 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:13:34.730424 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:34.730424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:34.730437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:34.730488 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:34.730798 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.731301 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.732012 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.732207 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.732264 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.734684 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:46067: BlockingRecv error: recv got EOF from 127.19.58.194:46067 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 02:13:34 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:34 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:34.856009 1042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:34.851644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:55971)
0808 02:13:34.851810 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:34.851821 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:34.851885 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:34.852001 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:34.852002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:34.852160 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:34.852209 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.852210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.852794 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.852795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.853304 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:34.853306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.853637 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.853638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:34.853676 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.854146 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:34.854150 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:34.855949 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":119,"threads_started":1}
W20260808 02:13:34.856174 1039 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:34.851721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:55971 (local address 127.19.58.254:42055)
0808 02:13:34.851784 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:34.851786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:34.851868 (+ 82us) server_negotiation.cc:407] Connection header received
0808 02:13:34.851939 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:34.851939 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:34.851952 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:34.851987 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:34.852256 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.852742 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.853379 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:34.853580 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:34.853630 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:34.856111 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:55971: BlockingRecv error: recv got EOF from 127.19.58.195:55971 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 02:13:35 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:35 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:35.608186 1044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:35.603922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:48035)
0808 02:13:35.604054 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:35.604064 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:35.604116 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:35.604239 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:35.604240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:35.604443 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:35.604495 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:35.604497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.605110 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.605111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:35.605660 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:35.605662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.605938 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.605939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:35.605977 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.606432 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:35.606436 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:35.608122 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":60,"threads_started":1}
W20260808 02:13:35.608292 1045 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:35.603966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:48035 (local address 127.19.58.254:42055)
0808 02:13:35.604083 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:35.604084 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:35.604104 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:13:35.604169 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:35.604169 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:35.604180 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:35.604227 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:35.604549 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.605056 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.605714 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.605894 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.605934 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.608214 (+ 2280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:48035: BlockingRecv error: recv got EOF from 127.19.58.193:48035 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
Aug 08 02:13:35 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:35 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:35.739410 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:35.735121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:48091)
0808 02:13:35.735254 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:35.735265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:35.735332 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:35.735468 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:35.735468 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:35.735643 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:35.735736 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:35.735738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.736430 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.736431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:35.736958 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:35.736959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.737255 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.737256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:35.737301 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.737697 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:35.737701 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:35.739349 (+ 1648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
W20260808 02:13:35.739533 1045 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:35.735170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:48091 (local address 127.19.58.254:42055)
0808 02:13:35.735225 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:35.735227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:35.735328 (+ 101us) server_negotiation.cc:407] Connection header received
0808 02:13:35.735387 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:35.735388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:35.735417 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:35.735456 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:35.735800 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.736349 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.737014 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.737196 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.737257 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.739472 (+ 2215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:48091: BlockingRecv error: recv got EOF from 127.19.58.194:48091 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 08 02:13:35 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:35 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:35.860826 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:35.856434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:57991)
0808 02:13:35.856557 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:35.856567 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:35.856623 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:35.856736 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:35.856736 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:35.856891 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:35.856986 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:35.856989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.857732 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.857733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:35.858264 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:35.858266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.858607 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.858609 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:35.858674 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.859077 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:35.859081 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:35.860748 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
W20260808 02:13:35.860898 1045 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:35.856508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:57991 (local address 127.19.58.254:42055)
0808 02:13:35.856557 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:35.856559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:35.856611 (+ 52us) server_negotiation.cc:407] Connection header received
0808 02:13:35.856676 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:35.856676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:35.856687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:35.856724 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:35.857066 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.857660 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.858341 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:35.858548 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:35.858602 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:35.860844 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:57991: BlockingRecv error: recv got EOF from 127.19.58.195:57991 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 02:13:36 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:36 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:36.612982 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:36.608638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:46277)
0808 02:13:36.608777 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:36.608786 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:36.608825 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:36.609039 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:36.609040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:36.609205 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:36.609258 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.609259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.609928 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.609929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.610475 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.610476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.610779 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.610779 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.610814 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.611221 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:36.611225 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:36.612920 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
W20260808 02:13:36.613095 1049 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:36.608760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:46277 (local address 127.19.58.254:42055)
0808 02:13:36.608926 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:36.608928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:36.608935 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:13:36.608955 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:36.608955 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:36.608966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:36.609029 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:36.609357 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.609864 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.610537 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.610729 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.610774 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.613024 (+ 2250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:46277: BlockingRecv error: recv got EOF from 127.19.58.193:46277 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
Aug 08 02:13:36 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:36 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:36.744385 1050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:36.739878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:53649)
0808 02:13:36.740024 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:36.740042 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:36.740116 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:36.740249 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:36.740250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:36.740436 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:36.740486 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.740489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.741126 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.741128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.741656 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.741657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.741997 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.741998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.742051 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.742538 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:36.742543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:36.744322 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
W20260808 02:13:36.744491 1049 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:36.739947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:53649 (local address 127.19.58.254:42055)
0808 02:13:36.740010 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:36.740019 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:36.740102 (+ 83us) server_negotiation.cc:407] Connection header received
0808 02:13:36.740183 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:36.740184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:36.740194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:36.740235 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:36.740561 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.741057 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.741729 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.741931 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.742008 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.744415 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:53649: BlockingRecv error: recv got EOF from 127.19.58.194:53649 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 02:13:36 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:36 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:36.865957 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:36.861278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:35977)
0808 02:13:36.861433 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:36.861444 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:36.861501 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:36.861615 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:36.861616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:36.861769 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:36.861818 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.861819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.862499 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.862500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.863078 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:36.863080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.863451 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.863452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:36.863493 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.863985 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:36.863990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:36.865896 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
W20260808 02:13:36.866076 1049 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:36.861335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:35977 (local address 127.19.58.254:42055)
0808 02:13:36.861384 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:36.861386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:36.861493 (+ 107us) server_negotiation.cc:407] Connection header received
0808 02:13:36.861544 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:36.861544 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:36.861556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:36.861604 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:36.861880 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.862425 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.863146 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:36.863384 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:36.863446 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:36.865991 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:35977: BlockingRecv error: recv got EOF from 127.19.58.195:35977 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 02:13:37 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:37 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:37.618110 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:37.613457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:38835)
0808 02:13:37.613623 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:37.613637 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:37.613707 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:37.613826 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:37.613827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:37.613990 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:37.614060 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.614061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.614895 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.614896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.615461 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.615464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.615854 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.615854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.615912 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.616341 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:37.616345 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:37.618026 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":75,"threads_started":1}
W20260808 02:13:37.618186 1057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:37.613591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:38835 (local address 127.19.58.254:42055)
0808 02:13:37.613718 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:37.613720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:37.613726 (+ 6us) server_negotiation.cc:407] Connection header received
0808 02:13:37.613750 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:37.613750 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:37.613759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:37.613812 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:37.614139 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.614820 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.615541 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.615797 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.615853 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.618124 (+ 2271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:38835: BlockingRecv error: recv got EOF from 127.19.58.193:38835 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
Aug 08 02:13:37 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:37 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:37.749425 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:37.744813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:49083)
0808 02:13:37.744950 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:37.744962 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:37.745032 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:37.745156 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:37.745156 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:37.745328 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:37.745389 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.745391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.746176 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.746177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.746741 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.746744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.747049 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.747049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.747091 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.747570 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:37.747575 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:37.749344 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
W20260808 02:13:37.749503 1057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:37.744900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:49083 (local address 127.19.58.254:42055)
0808 02:13:37.744968 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:37.744970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:37.745035 (+ 65us) server_negotiation.cc:407] Connection header received
0808 02:13:37.745077 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:37.745077 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:37.745088 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:37.745134 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:37.745453 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.746102 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.746806 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.746999 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.747045 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.749451 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:49083: BlockingRecv error: recv got EOF from 127.19.58.194:49083 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 02:13:37 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:37 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:37.870857 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:37.866448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:53089)
0808 02:13:37.866582 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:37.866592 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:37.866655 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:37.866767 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:37.866768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:37.866931 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:37.866980 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.866982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.867654 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.867655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.868247 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:37.868250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.868607 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.868608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:37.868652 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.869063 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:37.869067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:37.870785 (+ 1718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
W20260808 02:13:37.870936 1057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:37.866519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:53089 (local address 127.19.58.254:42055)
0808 02:13:37.866576 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:37.866578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:37.866646 (+ 68us) server_negotiation.cc:407] Connection header received
0808 02:13:37.866708 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:37.866709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:37.866721 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:37.866755 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:37.867026 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.867571 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.868327 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:37.868539 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:37.868595 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:37.870874 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:53089: BlockingRecv error: recv got EOF from 127.19.58.195:53089 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 02:13:38 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:38 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:38.623221 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:38.618593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:60393)
0808 02:13:38.618795 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:38.618808 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:38.618875 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:38.619046 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:38.619048 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:38.619275 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:38.619329 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.619332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.620060 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.620061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.620595 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.620596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.620934 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.620934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.620973 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.621390 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:38.621394 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:38.623127 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":72,"threads_started":1}
W20260808 02:13:38.623291 1061 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:38.618682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:60393 (local address 127.19.58.254:42055)
0808 02:13:38.618800 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:38.618803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:38.618871 (+ 68us) server_negotiation.cc:407] Connection header received
0808 02:13:38.618929 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:38.618930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:38.618950 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:38.619026 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:38.619452 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.619996 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.620659 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.620869 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.620931 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.623230 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:60393: BlockingRecv error: recv got EOF from 127.19.58.193:60393 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":64,"threads_started":1}
Aug 08 02:13:38 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:38 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:38.756742 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:38.749950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:36063)
0808 02:13:38.750079 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:38.750089 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:38.750149 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:38.750264 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:38.750264 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:38.750433 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:38.750486 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.750488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.751117 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.751117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.751645 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.751646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.751980 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.751981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.752021 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.754827 (+ 2806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:38.754832 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:38.756675 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
W20260808 02:13:38.756910 1061 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:38.750002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:36063 (local address 127.19.58.254:42055)
0808 02:13:38.750085 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:38.750088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:38.750146 (+ 58us) server_negotiation.cc:407] Connection header received
0808 02:13:38.750194 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:38.750195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:38.750206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:38.750252 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:38.750545 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.751059 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.751722 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.751925 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.751976 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.756781 (+ 4805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:36063: BlockingRecv error: recv got EOF from 127.19.58.194:36063 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 02:13:38 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:38 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:38.875785 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:38.871278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:57613)
0808 02:13:38.871408 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:38.871418 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:38.871475 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:38.871637 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:38.871638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:38.871820 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:38.871871 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.871872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.872491 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.872491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.873015 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:38.873017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.873362 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.873363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:38.873409 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.873849 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:38.873853 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:38.875725 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
W20260808 02:13:38.875880 1061 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:38.871350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:57613 (local address 127.19.58.254:42055)
0808 02:13:38.871401 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:38.871403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:38.871463 (+ 60us) server_negotiation.cc:407] Connection header received
0808 02:13:38.871528 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:38.871529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:38.871541 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:38.871581 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:38.871934 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.872433 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.873086 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:38.873277 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:38.873327 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:38.875822 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:57613: BlockingRecv error: recv got EOF from 127.19.58.195:57613 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 02:13:39 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:39 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:39.628201 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:39.623674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:42995)
0808 02:13:39.623831 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:39.623842 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:39.623903 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:39.624035 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:39.624035 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:39.624210 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:39.624266 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.624267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.624877 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.624877 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.625502 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.625504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.625811 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.625812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.625853 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.626331 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:39.626340 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:39.628118 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
W20260808 02:13:39.628269 1069 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:39.623757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:42995 (local address 127.19.58.254:42055)
0808 02:13:39.623862 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:39.623864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:39.623875 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:13:39.623959 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:39.623960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:39.623970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:39.624025 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:39.624323 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.624824 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.625557 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.625744 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.625800 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.628207 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:42995: BlockingRecv error: recv got EOF from 127.19.58.193:42995 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":48,"threads_started":1}
Aug 08 02:13:39 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:39 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:39.761385 1070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:39.757181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:46219)
0808 02:13:39.757300 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:39.757311 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:39.757371 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:39.757528 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:39.757529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:39.757693 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:39.757744 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.757745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.758337 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.758338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.758864 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.758865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.759152 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.759152 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.759188 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.759600 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:39.759605 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:39.761297 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
W20260808 02:13:39.761452 1069 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:39.757242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:46219 (local address 127.19.58.254:42055)
0808 02:13:39.757310 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:39.757312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:39.757365 (+ 53us) server_negotiation.cc:407] Connection header received
0808 02:13:39.757459 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:39.757459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:39.757470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:39.757516 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:39.757792 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.758286 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.758923 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.759103 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.759149 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.761399 (+ 2250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:46219: BlockingRecv error: recv got EOF from 127.19.58.194:46219 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 02:13:39 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:39 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:39.880581 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:39.876244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:50465)
0808 02:13:39.876400 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:39.876410 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:39.876471 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:39.876595 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:39.876595 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:39.876758 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:39.876808 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.876810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.877448 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.877449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.877981 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:39.877983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.878312 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.878312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:39.878351 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.878768 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:39.878772 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:39.880502 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
W20260808 02:13:39.880646 1069 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:39.876377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:50465 (local address 127.19.58.254:42055)
0808 02:13:39.876422 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:39.876424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:39.876456 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:13:39.876525 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:39.876525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:39.876536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:39.876584 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:39.876860 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.877381 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.878066 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:39.878255 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:39.878312 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:39.880595 (+ 2283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:50465: BlockingRecv error: recv got EOF from 127.19.58.195:50465 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 02:13:40 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:40 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:40.633152 1072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:40.628701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:34995)
0808 02:13:40.628843 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:40.628854 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:40.628908 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:40.629036 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:40.629036 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:40.629202 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:40.629252 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.629254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.629874 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.629875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.630442 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.630444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.630729 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.630730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.630765 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.631172 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:40.631176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:40.633079 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
W20260808 02:13:40.633244 1073 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:40.628776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:34995 (local address 127.19.58.254:42055)
0808 02:13:40.628887 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:40.628889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:40.628898 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:40.628960 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:40.628960 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:40.628971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:40.629026 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:40.629299 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.629807 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.630498 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.630683 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.630724 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.633173 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:34995: BlockingRecv error: recv got EOF from 127.19.58.193:34995 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
Aug 08 02:13:40 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:40 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:40.766652 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:40.761898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:46071)
0808 02:13:40.762052 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:40.762064 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:40.762133 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:40.762256 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:40.762257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:40.762472 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:40.762522 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.762524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.763422 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.763423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.764030 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.764032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.764379 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.764380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.764420 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.764842 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:40.764847 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:40.766580 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
W20260808 02:13:40.766729 1073 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:40.761999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:46071 (local address 127.19.58.254:42055)
0808 02:13:40.762073 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:40.762075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:40.762129 (+ 54us) server_negotiation.cc:407] Connection header received
0808 02:13:40.762187 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:40.762188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:40.762198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:40.762245 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:40.762616 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.763359 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.764100 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.764292 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.764340 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.766673 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:46071: BlockingRecv error: recv got EOF from 127.19.58.194:46071 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 02:13:40 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:40 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:40.885510 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:40.881068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:40679)
0808 02:13:40.881239 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:40.881252 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:40.881322 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:40.881448 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:40.881449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:40.881637 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:40.881682 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.881684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.882322 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.882323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.882856 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:40.882857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.883172 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.883173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:40.883228 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.883673 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:40.883678 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:40.885438 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
W20260808 02:13:40.885588 1073 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:40.881166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:40679 (local address 127.19.58.254:42055)
0808 02:13:40.881235 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:40.881237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:40.881303 (+ 66us) server_negotiation.cc:407] Connection header received
0808 02:13:40.881373 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:40.881374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:40.881387 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:40.881435 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:40.881755 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.882252 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.882931 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:40.883114 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:40.883168 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:40.885531 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:40679: BlockingRecv error: recv got EOF from 127.19.58.195:40679 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 02:13:41 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:41 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:41.638240 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:41.633606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:39901)
0808 02:13:41.633752 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:41.633762 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:41.633809 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:41.633934 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:41.633935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:41.634099 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:41.634150 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.634151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.634977 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.634978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.635579 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.635581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.635955 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.635955 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.635997 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.636475 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:41.636480 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:41.638169 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
W20260808 02:13:41.638432 1081 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:41.633669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:39901 (local address 127.19.58.254:42055)
0808 02:13:41.633783 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:41.633785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:41.633796 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:13:41.633851 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:41.633852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:41.633862 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:41.633925 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:41.634205 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.634912 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.635639 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.635899 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.635951 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.638355 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:39901: BlockingRecv error: recv got EOF from 127.19.58.193:39901 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
Aug 08 02:13:41 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:41 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:41.771282 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:41.767073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:33747)
0808 02:13:41.767222 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:41.767231 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:41.767287 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:41.767419 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:41.767419 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:41.767588 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:41.767638 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.767640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.768266 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.768267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.768803 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.768804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.769091 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.769092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.769128 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.769581 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:41.769586 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:41.771221 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
W20260808 02:13:41.771371 1081 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:41.767131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:33747 (local address 127.19.58.254:42055)
0808 02:13:41.767185 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:41.767186 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:41.767282 (+ 96us) server_negotiation.cc:407] Connection header received
0808 02:13:41.767340 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:41.767341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:41.767351 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:41.767393 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:41.767686 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.768202 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.768856 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.769043 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.769088 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.771308 (+ 2220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:33747: BlockingRecv error: recv got EOF from 127.19.58.194:33747 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 02:13:41 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:41 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:41.890321 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:41.885906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:35083)
0808 02:13:41.886060 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:41.886070 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:41.886138 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:41.886265 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:41.886265 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:41.886454 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:41.886501 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.886503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.887134 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.887136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.887675 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:41.887677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.888041 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.888042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:41.888098 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.888518 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:41.888522 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:41.890242 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
W20260808 02:13:41.890399 1081 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:41.885974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:35083 (local address 127.19.58.254:42055)
0808 02:13:41.886021 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:41.886023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:41.886128 (+ 105us) server_negotiation.cc:407] Connection header received
0808 02:13:41.886199 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:41.886200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:41.886213 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:41.886253 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:41.886573 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.887065 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.887789 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:41.887981 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:41.888036 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:41.890344 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:35083: BlockingRecv error: recv got EOF from 127.19.58.195:35083 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 02:13:42 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:42 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:42.642946 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:42.638652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:34067)
0808 02:13:42.638809 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:42.638821 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:42.638874 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:42.639004 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:42.639005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:42.639174 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:42.639222 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.639224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.639844 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.639845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.640385 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.640387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.640698 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.640699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.640736 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.641152 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:42.641156 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:42.642883 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
W20260808 02:13:42.643078 1087 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:42.638744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:34067 (local address 127.19.58.254:42055)
0808 02:13:42.638857 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:42.638859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:42.638867 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:42.638935 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:42.638935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:42.638946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:42.638991 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:42.639274 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.639790 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.640449 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.640650 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.640694 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.643006 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:34067: BlockingRecv error: recv got EOF from 127.19.58.193:34067 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
Aug 08 02:13:42 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:42 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:42.776228 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:42.771679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:33455)
0808 02:13:42.771834 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:42.771846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:42.771917 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:42.772054 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:42.772055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:42.772230 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:42.772277 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.772279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.772950 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.772951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.773480 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.773482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.773809 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.773810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.773864 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.774310 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:42.774314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:42.776148 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
W20260808 02:13:42.776306 1087 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:42.771780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:33455 (local address 127.19.58.254:42055)
0808 02:13:42.771836 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:42.771838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:42.771909 (+ 71us) server_negotiation.cc:407] Connection header received
0808 02:13:42.771980 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:42.771981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:42.771994 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:42.772042 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:42.772347 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.772863 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.773554 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.773751 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.773805 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.776253 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:33455: BlockingRecv error: recv got EOF from 127.19.58.194:33455 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 02:13:42 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:42 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:42.895187 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:42.890796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:43663)
0808 02:13:42.890923 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:42.890933 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:42.890992 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:42.891105 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:42.891106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:42.891272 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:42.891331 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.891333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.891975 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.891976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.892508 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:42.892509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.892888 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.892890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:42.892940 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.893357 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:42.893362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:42.895105 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
W20260808 02:13:42.895272 1087 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:42.890861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:43663 (local address 127.19.58.254:42055)
0808 02:13:42.890913 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:42.890915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:42.890979 (+ 64us) server_negotiation.cc:407] Connection header received
0808 02:13:42.891032 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:42.891032 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:42.891046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:42.891095 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:42.891386 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.891908 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.892599 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:42.892793 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:42.892851 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:42.895217 (+ 2366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:43663: BlockingRecv error: recv got EOF from 127.19.58.195:43663 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 02:13:43 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:43 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:43.647867 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:43.643432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:47245)
0808 02:13:43.643576 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:43.643588 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:43.643651 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:43.643817 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:43.643818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:43.644062 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:43.644118 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.644121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.644752 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.644753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.645289 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.645291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.645590 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.645591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.645629 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.646058 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:43.646062 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:43.647805 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
W20260808 02:13:43.647977 1093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:43.643505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:47245 (local address 127.19.58.254:42055)
0808 02:13:43.643617 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:43.643619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:43.643634 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:13:43.643720 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:43.643721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:43.643734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:43.643805 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:43.644183 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.644693 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.645344 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.645541 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.645587 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.647909 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:47245: BlockingRecv error: recv got EOF from 127.19.58.193:47245 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":61,"threads_started":1}
Aug 08 02:13:43 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:43 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:43.783653 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:43.776677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:53021)
0808 02:13:43.776825 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:43.776836 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:43.776912 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:43.777037 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:43.777038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:43.777194 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:43.777241 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.777243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.777837 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.777837 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.778401 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.778403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.778713 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.778714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.778752 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.781749 (+ 2997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:43.781754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:43.783560 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
W20260808 02:13:43.783743 1093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:43.776746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:53021 (local address 127.19.58.254:42055)
0808 02:13:43.776805 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:43.776806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:43.776907 (+ 101us) server_negotiation.cc:407] Connection header received
0808 02:13:43.776967 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:43.776967 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:43.776979 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:43.777028 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:43.777290 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.777779 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.778468 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.778659 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.778710 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.783661 (+ 4951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:53021: BlockingRecv error: recv got EOF from 127.19.58.194:53021 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 02:13:43 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:43 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.195@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:43.900380 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:43.895674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.195:59445)
0808 02:13:43.895853 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:43.895864 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:43.895928 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:43.896076 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:43.896076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:43.896246 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:43.896297 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.896299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.896935 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.896936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.897537 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:43.897540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.897872 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.897872 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:43.897915 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.898342 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:43.898346 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:43.900282 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":109,"threads_started":1}
W20260808 02:13:43.900431 1093 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:43.895765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:59445 (local address 127.19.58.254:42055)
0808 02:13:43.895825 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:43.895827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:43.895964 (+ 137us) server_negotiation.cc:407] Connection header received
0808 02:13:43.895992 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:43.895993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:43.896008 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:43.896065 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:43.896357 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.896852 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.897622 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:43.897815 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:43.897869 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:43.900379 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.195:59445: BlockingRecv error: recv got EOF from 127.19.58.195:59445 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 02:13:44 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:44 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.193@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:44.652803 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:44.648411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.193:59923)
0808 02:13:44.648542 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:44.648552 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:44.648599 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:44.648729 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:44.648729 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:44.648899 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:44.648951 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:44.648952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:44.649640 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:44.649641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:44.650203 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:44.650205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:44.650521 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:44.650522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:44.650556 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:44.650973 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:44.650977 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:44.652728 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
W20260808 02:13:44.652908 1101 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:44.648457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:59923 (local address 127.19.58.254:42055)
0808 02:13:44.648574 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:44.648576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:44.648587 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:13:44.648651 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:44.648651 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:44.648661 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:44.648719 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:44.649003 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:44.649570 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:44.650270 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:44.650468 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:44.650519 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:44.652819 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.193:59923: BlockingRecv error: recv got EOF from 127.19.58.193:59923 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":54,"threads_started":1}
Aug 08 02:13:44 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 02:13:44 dist-test-slave-1l3l krb5kdc[32016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.58.194@KRBTEST.COM for oryx/127.19.58.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 02:13:44.788631 1103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:44.784175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42055 (local address 127.19.58.194:44527)
0808 02:13:44.784294 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:44.784306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:44.784386 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:44.784499 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:44.784499 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:44.784653 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:44.784703 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:44.784704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:44.785318 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:44.785319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:44.785991 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:44.785994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:44.786319 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:44.786319 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:44.786358 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:44.786820 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:44.786824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:44.788572 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.58.254:42055: Server oryx/127.19.58.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":82,"threads_started":1}
W20260808 02:13:44.788731 1101 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:44.784231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:44527 (local address 127.19.58.254:42055)
0808 02:13:44.784328 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:44.784330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:44.784406 (+ 76us) server_negotiation.cc:407] Connection header received
0808 02:13:44.784434 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:44.784435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:44.784445 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:44.784496 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:44.784748 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:44.785250 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:44.786066 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:44.786260 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:44.786313 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:44.788665 (+ 2352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.58.194:44527: BlockingRecv error: recv got EOF from 127.19.58.194:44527 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 02:13:44.813455 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 409
I20260808 02:13:44.828073 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 627
I20260808 02:13:44.837309 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 810
I20260808 02:13:44.844048 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 312
2026-08-08T02:13:44Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17492 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T02:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:44Z Disabled control of system clock
I20260808 02:13:44.878798 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:37365
--webserver_interface=127.19.58.254
--webserver_port=0
--builtin_ntp_servers=127.19.58.212:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:37365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 02:13:44.959223 1107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:44.959405 1107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:44.959424 1107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:44.960526 1107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:44.960561 1107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:44.960572 1107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:44.960582 1107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:44.960595 1107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:44.961834 1107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:37365
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:37365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:44.962018 1107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:44.962244 1107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:44.966989 1115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:44.966998 1118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:44.967041 1107 server_base.cc:1034] running on GCE node
W20260808 02:13:44.967024 1114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:44.967514 1107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:44.967785 1107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:44.968926 1107 hybrid_clock.cc:648] HybridClock initialized: now 1786155224968913 us; error 31 us; skew 500 ppm
I20260808 02:13:44.970039 1107 webserver.cc:460] Webserver started at http://127.19.58.254:40057/ using document root <none> and password file <none>
I20260808 02:13:44.970256 1107 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:44.970299 1107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:44.970428 1107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:44.971459 1107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "d2de015f37574296b2fb8dd47d48a594"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-1l3l"
server_key: "6e51890563ca8f5171f86f5c9ba5f9b1"
server_key_iv: "316daa2307c2a1055d7d7ae83caf277f"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.971854 1107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "d2de015f37574296b2fb8dd47d48a594"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-1l3l"
server_key: "6e51890563ca8f5171f86f5c9ba5f9b1"
server_key_iv: "316daa2307c2a1055d7d7ae83caf277f"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.973089 1107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:44.973795 1124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:44.973973 1107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:44.974051 1107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "d2de015f37574296b2fb8dd47d48a594"
format_stamp: "Formatted at 2026-08-08 02:13:44 on dist-test-slave-1l3l"
server_key: "6e51890563ca8f5171f86f5c9ba5f9b1"
server_key_iv: "316daa2307c2a1055d7d7ae83caf277f"
server_key_version: "encryptionkey@0"
I20260808 02:13:44.974119 1107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:44.990880 1107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:44.991132 1107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:44.991254 1107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:44.995163 1107 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:37365
I20260808 02:13:44.995196 1189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:37365 every 8 connection(s)
I20260808 02:13:44.995512 1107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:44.996173 1190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:44.998672 1190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594: Bootstrap starting.
I20260808 02:13:44.999217 1190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:44.999498 1190 log.cc:824] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:45.000171 1190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594: No bootstrap required, opened a new log
I20260808 02:13:45.001776 1190 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de015f37574296b2fb8dd47d48a594" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37365 } }
I20260808 02:13:45.001912 1190 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:45.001935 1190 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2de015f37574296b2fb8dd47d48a594, State: Initialized, Role: FOLLOWER
I20260808 02:13:45.002064 1190 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de015f37574296b2fb8dd47d48a594" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37365 } }
I20260808 02:13:45.002131 1190 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:45.002151 1190 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:45.002179 1190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:45.002713 1190 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de015f37574296b2fb8dd47d48a594" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37365 } }
I20260808 02:13:45.002806 1190 leader_election.cc:302] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2de015f37574296b2fb8dd47d48a594; no voters:
I20260808 02:13:45.002835 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 1107
I20260808 02:13:45.002924 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:45.002956 1190 leader_election.cc:288] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:45.002993 1193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:45.003129 19691 external_mini_cluster.cc:1428] Setting key 447ba32f49e0a57b5bd24576b18fd39b
I20260808 02:13:45.003155 1193 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [term 1 LEADER]: Becoming Leader. State: Replica: d2de015f37574296b2fb8dd47d48a594, State: Running, Role: LEADER
I20260808 02:13:45.003204 1190 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:45.003257 1193 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de015f37574296b2fb8dd47d48a594" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37365 } }
I20260808 02:13:45.003901 1194 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2de015f37574296b2fb8dd47d48a594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de015f37574296b2fb8dd47d48a594" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37365 } } }
I20260808 02:13:45.003984 1194 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:45.004112 1195 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2de015f37574296b2fb8dd47d48a594. Latest consensus state: current_term: 1 leader_uuid: "d2de015f37574296b2fb8dd47d48a594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de015f37574296b2fb8dd47d48a594" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37365 } } }
I20260808 02:13:45.004189 1195 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:45.004468 1205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:45.004998 1205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:45.007382 1205 catalog_manager.cc:1347] Generated new cluster ID: 12f6b17ba1ab435eae5262dafd7df528
I20260808 02:13:45.007439 1205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:45.009336 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.003572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.19.58.254:37365)
0808 02:13:45.004406 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:45.004415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:45.004449 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:13:45.004561 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:45.004568 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:45.004635 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:45.004780 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:45.005039 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.006236 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.007313 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.007792 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.007909 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:45.008675 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:45.008680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:45.008680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:45.008857 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:45.008891 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:45.008902 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:45.008959 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:45.009034 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 02:13:45.009123 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":170,"threads_started":1}
I20260808 02:13:45.053188 1205 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:45.053925 1205 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:45.062943 1205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2de015f37574296b2fb8dd47d48a594: Generated new TSK 0
I20260808 02:13:45.063138 1205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:45.067641 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37365
--builtin_ntp_servers=127.19.58.212:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 02:13:45.151630 1218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:45.151876 1218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:45.151901 1218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:45.153041 1218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:45.153081 1218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:45.153105 1218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:45.154425 1218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:45.154652 1218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:45.154937 1218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:45.159886 1232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.159894 1228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.159916 1227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:45.160185 1218 server_base.cc:1034] running on GCE node
I20260808 02:13:45.160385 1218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:45.160604 1218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:45.161736 1218 hybrid_clock.cc:648] HybridClock initialized: now 1786155225161724 us; error 35 us; skew 500 ppm
I20260808 02:13:45.162874 1218 webserver.cc:460] Webserver started at http://127.19.58.193:37407/ using document root <none> and password file <none>
I20260808 02:13:45.163115 1218 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:45.163165 1218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:45.163291 1218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:45.164330 1218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "51e3da216e284742a586f3663b6939e8"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "98ce949d30a5a61753bf37b8ad369133"
server_key_iv: "69a7d6763340c9b62e692a26fa288e47"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.164650 1218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "51e3da216e284742a586f3663b6939e8"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "98ce949d30a5a61753bf37b8ad369133"
server_key_iv: "69a7d6763340c9b62e692a26fa288e47"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.165817 1218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:45.166435 1238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.166584 1218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:45.166662 1218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "51e3da216e284742a586f3663b6939e8"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "98ce949d30a5a61753bf37b8ad369133"
server_key_iv: "69a7d6763340c9b62e692a26fa288e47"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.166723 1218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:45.173367 1218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:45.173596 1218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:45.173713 1218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:45.173906 1218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:45.174182 1218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:45.174216 1218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.174242 1218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:45.174266 1218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.180418 1218 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:35627
I20260808 02:13:45.180451 1396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:35627 every 8 connection(s)
I20260808 02:13:45.180727 1218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:45.181725 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 1218
I20260808 02:13:45.181808 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:45.181983 19691 external_mini_cluster.cc:1428] Setting key b2e4beb71a8f8c3d79951d92871cbb19
I20260808 02:13:45.182855 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37365
--builtin_ntp_servers=127.19.58.212:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 02:13:45.185917 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.181344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:51871 (local address 127.19.58.254:37365)
0808 02:13:45.181398 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:45.181400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:45.181631 (+ 231us) server_negotiation.cc:407] Connection header received
0808 02:13:45.182123 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:45.182125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:45.182140 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:45.182201 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:45.182861 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.183525 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.184539 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.184807 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.184861 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:45.185685 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:45.185686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:45.185686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:45.185692 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:45.185705 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:45.185709 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:45.185742 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:45.185827 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 02:13:45.185856 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 02:13:45.186132 1402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.181323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:37365 (local address 127.19.58.193:51871)
0808 02:13:45.181501 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:45.181537 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:45.182029 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:45.182267 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:45.182271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:45.182280 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:45.182775 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:45.182780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.183616 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.183617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:45.184449 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:45.184451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.184882 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.184883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:45.185041 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:45.185533 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:45.185533 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:45.185568 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:45.185594 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:45.185757 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:45.185762 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:45.185765 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:45.185827 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 02:13:45.185931 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260808 02:13:45.186569 1400 heartbeater.cc:347] Connected to a master server at 127.19.58.254:37365
I20260808 02:13:45.186654 1400 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:45.186817 1400 heartbeater.cc:511] Master 127.19.58.254:37365 requested a full tablet report, sending...
I20260808 02:13:45.187309 1147 ts_manager.cc:194] Registered new tserver with Master: 51e3da216e284742a586f3663b6939e8 (127.19.58.193:35627)
I20260808 02:13:45.187973 1147 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.193:51871
W20260808 02:13:45.263207 1403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:45.263458 1403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:45.263492 1403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:45.264645 1403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:45.264684 1403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:45.264716 1403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:45.266009 1403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:45.266227 1403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:45.266515 1403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:45.271271 1416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.271296 1420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.271265 1418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:45.271485 1403 server_base.cc:1034] running on GCE node
I20260808 02:13:45.271689 1403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:45.271940 1403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:45.273079 1403 hybrid_clock.cc:648] HybridClock initialized: now 1786155225273062 us; error 34 us; skew 500 ppm
I20260808 02:13:45.274214 1403 webserver.cc:460] Webserver started at http://127.19.58.194:33675/ using document root <none> and password file <none>
I20260808 02:13:45.274432 1403 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:45.274484 1403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:45.274613 1403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:45.275662 1403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "f5226044511b4b08a09be60eeff04609"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "6687c7f6c286d294e74b0ea6ea5354ff"
server_key_iv: "3a532bb6009b267f64830882beb69f62"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.276047 1403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "f5226044511b4b08a09be60eeff04609"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "6687c7f6c286d294e74b0ea6ea5354ff"
server_key_iv: "3a532bb6009b267f64830882beb69f62"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.277305 1403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:45.277984 1431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.278157 1403 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:45.278236 1403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "f5226044511b4b08a09be60eeff04609"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "6687c7f6c286d294e74b0ea6ea5354ff"
server_key_iv: "3a532bb6009b267f64830882beb69f62"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.278301 1403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:45.298265 1403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:45.298517 1403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:45.298640 1403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:45.298899 1403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:45.299203 1403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:45.299237 1403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.299265 1403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:45.299283 1403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.305306 1403 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:44091
I20260808 02:13:45.305371 1602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:44091 every 8 connection(s)
I20260808 02:13:45.305691 1403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:45.308547 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 1403
I20260808 02:13:45.308619 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:45.308776 19691 external_mini_cluster.cc:1428] Setting key 4cadeddce8acf8becd61248cc0797ed5
I20260808 02:13:45.309674 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37365
--builtin_ntp_servers=127.19.58.212:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 02:13:45.310786 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.306297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:43211 (local address 127.19.58.254:37365)
0808 02:13:45.306344 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:45.306346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:45.306571 (+ 225us) server_negotiation.cc:407] Connection header received
0808 02:13:45.307076 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:45.307077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:45.307090 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:45.307137 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:45.307806 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.308301 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.309500 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.309713 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.309767 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:45.310537 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:45.310538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:45.310539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:45.310543 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:45.310558 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:45.310563 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:45.310607 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:45.310679 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 02:13:45.310708 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:13:45.310999 1606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.306278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:37365 (local address 127.19.58.194:43211)
0808 02:13:45.306450 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:45.306491 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:45.306970 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:45.307213 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:45.307217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:45.307226 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:45.307689 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:45.307694 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.308403 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.308404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:45.309434 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:45.309437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.309785 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.309786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:45.309924 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:45.310412 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:45.310413 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:45.310439 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:45.310460 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:45.310616 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:45.310621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:45.310624 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:45.310683 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:13:45.310776 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260808 02:13:45.311362 1604 heartbeater.cc:347] Connected to a master server at 127.19.58.254:37365
I20260808 02:13:45.311443 1604 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:45.311605 1604 heartbeater.cc:511] Master 127.19.58.254:37365 requested a full tablet report, sending...
I20260808 02:13:45.311954 1147 ts_manager.cc:194] Registered new tserver with Master: f5226044511b4b08a09be60eeff04609 (127.19.58.194:44091)
I20260808 02:13:45.312311 1147 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.194:43211
W20260808 02:13:45.389211 1608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:45.389390 1608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:45.389410 1608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:45.390473 1608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:45.390506 1608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:45.390527 1608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:45.391830 1608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:45.392016 1608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:45.392247 1608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:45.396973 1623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.396983 1622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:45.397200 1608 server_base.cc:1034] running on GCE node
W20260808 02:13:45.396983 1625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:45.397466 1608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:45.397693 1608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:45.398815 1608 hybrid_clock.cc:648] HybridClock initialized: now 1786155225398805 us; error 32 us; skew 500 ppm
I20260808 02:13:45.400050 1608 webserver.cc:460] Webserver started at http://127.19.58.195:42013/ using document root <none> and password file <none>
I20260808 02:13:45.400256 1608 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:45.400301 1608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:45.400406 1608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:45.401402 1608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "d825bcfb029745309e43e5b76bb772c2"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "92cf17aa0ca0544ec9e05b12e46e9f71"
server_key_iv: "9136c359cdd578198575787fccb9abf3"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.401707 1608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "d825bcfb029745309e43e5b76bb772c2"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "92cf17aa0ca0544ec9e05b12e46e9f71"
server_key_iv: "9136c359cdd578198575787fccb9abf3"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.402926 1608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:45.403614 1632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.403786 1608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:45.403858 1608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "d825bcfb029745309e43e5b76bb772c2"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "92cf17aa0ca0544ec9e05b12e46e9f71"
server_key_iv: "9136c359cdd578198575787fccb9abf3"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.403929 1608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:45.413900 1608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:45.414124 1608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:45.414233 1608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:45.414407 1608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:45.414690 1608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:45.414722 1608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.414748 1608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:45.414765 1608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.420209 1608 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:33241
I20260808 02:13:45.420264 1793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:33241 every 8 connection(s)
I20260808 02:13:45.420589 1608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:45.425041 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 1608
I20260808 02:13:45.425083 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.421224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:52839 (local address 127.19.58.254:37365)
0808 02:13:45.421276 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:45.421278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:45.421490 (+ 212us) server_negotiation.cc:407] Connection header received
0808 02:13:45.421920 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:45.421921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:45.421930 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:45.421982 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:45.422558 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.423040 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.423970 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.424150 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.424193 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:45.424850 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:45.424851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:45.424852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:45.424856 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:45.424870 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:45.424874 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:45.424914 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:45.425002 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 02:13:45.425025 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 02:13:45.425135 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:45.425328 19691 external_mini_cluster.cc:1428] Setting key b8e53d80268a7e64e3ca7138ce44b55b
I20260808 02:13:45.425348 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.421205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:37365 (local address 127.19.58.195:52839)
0808 02:13:45.421376 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:45.421415 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:45.421839 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:45.422039 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:45.422042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:45.422049 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:45.422498 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:45.422502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.423095 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.423096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:45.423903 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:45.423906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:45.424202 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:45.424203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:45.424301 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:45.424727 (+ 426us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:45.424727 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:45.424753 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:45.424773 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:45.424928 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:45.424933 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:45.424936 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:45.425007 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:13:45.425108 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260808 02:13:45.425863 1794 heartbeater.cc:347] Connected to a master server at 127.19.58.254:37365
I20260808 02:13:45.425961 1794 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:45.426098 1794 heartbeater.cc:511] Master 127.19.58.254:37365 requested a full tablet report, sending...
I20260808 02:13:45.426409 1147 ts_manager.cc:194] Registered new tserver with Master: d825bcfb029745309e43e5b76bb772c2 (127.19.58.195:33241)
I20260808 02:13:45.426749 1147 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.195:52839
I20260808 02:13:45.437044 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:13:45.438723 1197 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:45.438293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.19.58.254:37365)
0808 02:13:45.438329 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:45.438331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:45.438476 (+ 145us) server_negotiation.cc:407] Connection header received
0808 02:13:45.438499 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:45.438500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:45.438513 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:45.438565 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:45.438673 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33450: BlockingRecv error: recv got EOF from 127.0.0.1:33450 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 02:13:45.438786 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 1218
I20260808 02:13:45.451349 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 1403
I20260808 02:13:45.463356 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 1608
I20260808 02:13:45.471345 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 1107
2026-08-08T02:13:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (623 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T02:13:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:45Z Disabled control of system clock
I20260808 02:13:45.502107 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:37179
--webserver_interface=127.19.58.254
--webserver_port=0
--builtin_ntp_servers=127.19.58.212:39447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:37179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:13:45.581488 1807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:45.581673 1807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:45.581692 1807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:45.582746 1807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:45.582782 1807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:45.582793 1807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:45.582810 1807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:45.582821 1807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:45.584143 1807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:37179
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:37179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:45.584338 1807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:45.584556 1807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:45.589037 1816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.589059 1818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.589061 1814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:45.589212 1807 server_base.cc:1034] running on GCE node
I20260808 02:13:45.589481 1807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:45.589707 1807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:45.590837 1807 hybrid_clock.cc:648] HybridClock initialized: now 1786155225590815 us; error 38 us; skew 500 ppm
I20260808 02:13:45.591980 1807 webserver.cc:460] Webserver started at http://127.19.58.254:43451/ using document root <none> and password file <none>
I20260808 02:13:45.592192 1807 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:45.592242 1807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:45.592362 1807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:45.593343 1807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "2ad73e6c7ead46f8acc7778abc2ae330"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "5762ee84e622cdd784bb2b77850175c4"
server_key_iv: "cc7a88308d9b2cccf3fc703e337fd001"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.593659 1807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "2ad73e6c7ead46f8acc7778abc2ae330"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "5762ee84e622cdd784bb2b77850175c4"
server_key_iv: "cc7a88308d9b2cccf3fc703e337fd001"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.594810 1807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:45.595482 1824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.595630 1807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:45.595731 1807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "2ad73e6c7ead46f8acc7778abc2ae330"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "5762ee84e622cdd784bb2b77850175c4"
server_key_iv: "cc7a88308d9b2cccf3fc703e337fd001"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.595791 1807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:45.598884 1807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:45.599116 1807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:45.599233 1807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:45.602813 1807 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:37179
I20260808 02:13:45.602867 1905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:37179 every 8 connection(s)
I20260808 02:13:45.603153 1807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:45.603788 1906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:45.606114 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 1807
I20260808 02:13:45.606211 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:45.606387 19691 external_mini_cluster.cc:1428] Setting key 7d48c4aecc08e7fdae91015daf2b5fee
I20260808 02:13:45.606405 1906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330: Bootstrap starting.
I20260808 02:13:45.607061 1906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:45.607383 1906 log.cc:824] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:45.608136 1906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330: No bootstrap required, opened a new log
I20260808 02:13:45.608847 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.606866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50464 (local address 127.19.58.254:37179)
0808 02:13:45.607061 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:45.607069 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:45.607110 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:13:45.607636 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:45.607644 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:45.607683 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:45.607846 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:45.608449 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:45.608452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:45.608452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:45.608456 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:45.608494 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:45.608504 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:45.608563 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:45.608624 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 02:13:45.608669 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260808 02:13:45.609737 1906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad73e6c7ead46f8acc7778abc2ae330" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37179 } }
I20260808 02:13:45.609869 1906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:45.609903 1906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad73e6c7ead46f8acc7778abc2ae330, State: Initialized, Role: FOLLOWER
I20260808 02:13:45.610018 1906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad73e6c7ead46f8acc7778abc2ae330" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37179 } }
I20260808 02:13:45.610091 1906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:45.610121 1906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:45.610167 1906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:45.610913 1906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad73e6c7ead46f8acc7778abc2ae330" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37179 } }
I20260808 02:13:45.611006 1906 leader_election.cc:302] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ad73e6c7ead46f8acc7778abc2ae330; no voters:
I20260808 02:13:45.611155 1906 leader_election.cc:288] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:45.611195 1914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:45.611327 1914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [term 1 LEADER]: Becoming Leader. State: Replica: 2ad73e6c7ead46f8acc7778abc2ae330, State: Running, Role: LEADER
I20260808 02:13:45.611393 1906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:45.611431 1914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad73e6c7ead46f8acc7778abc2ae330" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37179 } }
I20260808 02:13:45.611804 1914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ad73e6c7ead46f8acc7778abc2ae330. Latest consensus state: current_term: 1 leader_uuid: "2ad73e6c7ead46f8acc7778abc2ae330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad73e6c7ead46f8acc7778abc2ae330" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37179 } } }
I20260808 02:13:45.611889 1914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:45.611812 1917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ad73e6c7ead46f8acc7778abc2ae330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad73e6c7ead46f8acc7778abc2ae330" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 37179 } } }
I20260808 02:13:45.611969 1917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330 [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:45.612361 1928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:45.612838 1928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:45.614190 1928 catalog_manager.cc:1347] Generated new cluster ID: f4498e257f53412d8005c59a5b8abf7e
I20260808 02:13:45.614246 1928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:45.629688 1928 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:45.629835 1928 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:45.634071 1928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ad73e6c7ead46f8acc7778abc2ae330: Generated new TSK 0
I20260808 02:13:45.634243 1928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:13:45.641464 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37179
--builtin_ntp_servers=127.19.58.212:39447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:13:45.722213 1943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:45.722404 1943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:45.722424 1943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:45.723543 1943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:45.723582 1943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:45.723605 1943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:45.724921 1943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:45.725126 1943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:45.725397 1943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:45.730122 1950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.730119 1952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.730171 1949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:45.730509 1943 server_base.cc:1034] running on GCE node
I20260808 02:13:45.730705 1943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:45.730953 1943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:45.732115 1943 hybrid_clock.cc:648] HybridClock initialized: now 1786155225732102 us; error 30 us; skew 500 ppm
I20260808 02:13:45.733485 1943 webserver.cc:460] Webserver started at http://127.19.58.193:44651/ using document root <none> and password file <none>
I20260808 02:13:45.733717 1943 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:45.733771 1943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:45.733896 1943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:45.735196 1943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "7c4eadb124ee48cb96b82d555a085497"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "a5a93002880f6bf97b55b90b906f0f2c"
server_key_iv: "111da3a81a56ba90f2336c963cf81ea0"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.735587 1943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "7c4eadb124ee48cb96b82d555a085497"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "a5a93002880f6bf97b55b90b906f0f2c"
server_key_iv: "111da3a81a56ba90f2336c963cf81ea0"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.737124 1943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:45.737905 1961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.738076 1943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:45.738175 1943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "7c4eadb124ee48cb96b82d555a085497"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "a5a93002880f6bf97b55b90b906f0f2c"
server_key_iv: "111da3a81a56ba90f2336c963cf81ea0"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.738250 1943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:45.742573 1943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:45.742852 1943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:45.742950 1943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:45.743101 1943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:45.743335 1943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:45.743366 1943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.743383 1943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:45.743400 1943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.748776 1943 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:44761
I20260808 02:13:45.748812 2134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:44761 every 8 connection(s)
I20260808 02:13:45.749076 1943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:13:45.751580 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.749665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:44049 (local address 127.19.58.254:37179)
0808 02:13:45.749711 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:45.749714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:45.749942 (+ 228us) server_negotiation.cc:407] Connection header received
0808 02:13:45.750352 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:45.750353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:45.750359 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:45.750427 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:45.751405 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:45.751406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:45.751406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:45.751410 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:45.751423 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:45.751427 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:45.751473 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:45.751517 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 02:13:45.751528 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:13:45.751761 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.749668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:37179 (local address 127.19.58.193:44049)
0808 02:13:45.749834 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:45.749864 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:45.750268 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:45.750476 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:45.750478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:45.750486 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:45.751265 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:45.751266 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:45.751290 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:45.751324 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:45.751471 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:45.751473 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:45.751476 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:13:45.751521 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 02:13:45.751565 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260808 02:13:45.752197 2135 heartbeater.cc:347] Connected to a master server at 127.19.58.254:37179
I20260808 02:13:45.752275 2135 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:45.752415 2135 heartbeater.cc:511] Master 127.19.58.254:37179 requested a full tablet report, sending...
I20260808 02:13:45.752844 1853 ts_manager.cc:194] Registered new tserver with Master: 7c4eadb124ee48cb96b82d555a085497 (127.19.58.193:44761)
I20260808 02:13:45.756733 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 1943
I20260808 02:13:45.756814 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:45.757004 19691 external_mini_cluster.cc:1428] Setting key 8f831a28a22541d3517f9321ba452506
I20260808 02:13:45.757810 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37179
--builtin_ntp_servers=127.19.58.212:39447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 02:13:45.837409 2138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:45.837597 2138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:45.837618 2138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:45.838733 2138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:45.838771 2138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:45.838796 2138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:45.840157 2138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:45.840360 2138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:45.840621 2138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:13:45.845424 2138 server_base.cc:1034] running on GCE node
W20260808 02:13:45.845439 2146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.845551 2149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.845453 2147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:45.845904 2138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:45.846107 2138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:45.847232 2138 hybrid_clock.cc:648] HybridClock initialized: now 1786155225847215 us; error 34 us; skew 500 ppm
I20260808 02:13:45.848477 2138 webserver.cc:460] Webserver started at http://127.19.58.194:45011/ using document root <none> and password file <none>
I20260808 02:13:45.848677 2138 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:45.848721 2138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:45.848841 2138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:45.849932 2138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "2fe57b63d9e846ec90946fe002f101df"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "a9504e58b00cf3bd1a6473d271302981"
server_key_iv: "e5038283870ab06f88902732d6a6a54c"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.850244 2138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "2fe57b63d9e846ec90946fe002f101df"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "a9504e58b00cf3bd1a6473d271302981"
server_key_iv: "e5038283870ab06f88902732d6a6a54c"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.851413 2138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:13:45.852075 2156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.852226 2138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.852309 2138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "2fe57b63d9e846ec90946fe002f101df"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "a9504e58b00cf3bd1a6473d271302981"
server_key_iv: "e5038283870ab06f88902732d6a6a54c"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.852361 2138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:45.855422 2138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:45.855647 2138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:45.855783 2138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:45.855976 2138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:45.856252 2138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:45.856283 2138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.856312 2138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:45.856334 2138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.861536 2138 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:36639
I20260808 02:13:45.861595 2326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:36639 every 8 connection(s)
I20260808 02:13:45.861869 2138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:45.862579 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 2138
I20260808 02:13:45.862689 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:45.862910 19691 external_mini_cluster.cc:1428] Setting key 837a64729a26d997304e59f85b1a03ab
I20260808 02:13:45.863867 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37179
--builtin_ntp_servers=127.19.58.212:39447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 02:13:45.864961 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.863111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:40363 (local address 127.19.58.254:37179)
0808 02:13:45.863178 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:45.863180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:45.863190 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:13:45.863332 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:45.863332 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:45.863337 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:45.863397 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:45.864738 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:45.864739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:45.864739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:45.864743 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:45.864758 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:45.864761 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:45.864799 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:45.864899 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 02:13:45.864910 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:13:45.865159 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.862470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:37179 (local address 127.19.58.194:40363)
0808 02:13:45.862651 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:45.862698 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:45.863242 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:45.863471 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:45.863475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:45.863485 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:45.864573 (+ 1088us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:45.864574 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:45.864610 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:45.864642 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:45.864820 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:45.864825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:45.864827 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:13:45.864898 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:13:45.864958 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260808 02:13:45.865872 2327 heartbeater.cc:347] Connected to a master server at 127.19.58.254:37179
I20260808 02:13:45.865962 2327 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:45.866097 2327 heartbeater.cc:511] Master 127.19.58.254:37179 requested a full tablet report, sending...
I20260808 02:13:45.866418 1853 ts_manager.cc:194] Registered new tserver with Master: 2fe57b63d9e846ec90946fe002f101df (127.19.58.194:36639)
W20260808 02:13:45.942826 2333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:45.943037 2333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:45.943064 2333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:45.944208 2333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:45.944247 2333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:45.944271 2333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:45.945549 2333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:37179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:45.945722 2333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:45.945966 2333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:45.950713 2342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.950717 2343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:45.950722 2345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:45.950883 2333 server_base.cc:1034] running on GCE node
I20260808 02:13:45.951207 2333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:45.951442 2333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:45.952682 2333 hybrid_clock.cc:648] HybridClock initialized: now 1786155225952667 us; error 31 us; skew 500 ppm
I20260808 02:13:45.953879 2333 webserver.cc:460] Webserver started at http://127.19.58.195:44317/ using document root <none> and password file <none>
I20260808 02:13:45.954094 2333 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:45.954138 2333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:45.954275 2333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:45.955413 2333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "1be4814de6e64fcc93eacd9acebf6db8"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "3b300839951d51724ee09b326f3d7496"
server_key_iv: "c2a68de71335c517d8358b43a482b495"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.955804 2333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "1be4814de6e64fcc93eacd9acebf6db8"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "3b300839951d51724ee09b326f3d7496"
server_key_iv: "c2a68de71335c517d8358b43a482b495"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.957000 2333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:13:45.957600 2351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.957738 2333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:45.957811 2333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "1be4814de6e64fcc93eacd9acebf6db8"
format_stamp: "Formatted at 2026-08-08 02:13:45 on dist-test-slave-1l3l"
server_key: "3b300839951d51724ee09b326f3d7496"
server_key_iv: "c2a68de71335c517d8358b43a482b495"
server_key_version: "encryptionkey@0"
I20260808 02:13:45.957876 2333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:45.961110 2333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:45.961330 2333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:45.961452 2333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:45.961634 2333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:45.961905 2333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:45.961946 2333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.961979 2333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:45.962002 2333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:45.967262 2333 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:41179
I20260808 02:13:45.967319 2507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:41179 every 8 connection(s)
I20260808 02:13:45.967576 2333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:45.968892 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 2333
I20260808 02:13:45.969002 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:45.969197 19691 external_mini_cluster.cc:1428] Setting key 111a2213bf377b5864cab11845175ebc
I20260808 02:13:45.970656 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.968391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:43125 (local address 127.19.58.254:37179)
0808 02:13:45.968463 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:45.968465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:45.968622 (+ 157us) server_negotiation.cc:407] Connection header received
0808 02:13:45.969193 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:45.969195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:45.969201 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:45.969254 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:45.970482 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:45.970483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:45.970483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:13:45.970487 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:45.970501 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:13:45.970504 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:45.970550 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:45.970594 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 02:13:45.970604 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 02:13:45.970813 2512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:45.968298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:37179 (local address 127.19.58.195:43125)
0808 02:13:45.968516 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:45.968546 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:45.969086 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:45.969361 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:45.969366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:45.969376 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:45.970351 (+ 975us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:13:45.970352 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:45.970377 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:13:45.970402 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:45.970549 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:45.970551 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:45.970553 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:13:45.970599 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:13:45.970641 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260808 02:13:45.971184 2508 heartbeater.cc:347] Connected to a master server at 127.19.58.254:37179
I20260808 02:13:45.971264 2508 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:45.971380 2508 heartbeater.cc:511] Master 127.19.58.254:37179 requested a full tablet report, sending...
I20260808 02:13:45.971654 1853 ts_manager.cc:194] Registered new tserver with Master: 1be4814de6e64fcc93eacd9acebf6db8 (127.19.58.195:41179)
I20260808 02:13:45.980744 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 02:13:45.982367 1911 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:13:45.982025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50474 (local address 127.19.58.254:37179)
0808 02:13:45.982081 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:45.982083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:45.982133 (+ 50us) server_negotiation.cc:407] Connection header received
0808 02:13:45.982170 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:45.982171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:45.982175 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:45.982222 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:45.982315 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50474: BlockingRecv error: recv got EOF from 127.0.0.1:50474 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:13:45.982450 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 1943
I20260808 02:13:45.996393 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 2138
I20260808 02:13:46.004285 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 2333
I20260808 02:13:46.028702 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 1807
2026-08-08T02:13:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (557 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:46 dist-test-slave-1l3l krb5kdc[2522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:46 dist-test-slave-1l3l krb5kdc[2522](info): set up 2 sockets
Aug 08 02:13:46 dist-test-slave-1l3l krb5kdc[2522](info): commencing operation
krb5kdc: starting...
W20260808 02:13:47.147284 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:47 dist-test-slave-1l3l krb5kdc[2522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:13:47.258514 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:43409
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:43409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:47.337337 2537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:47.337525 2537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:47.337554 2537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:47.338662 2537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:13:47.338702 2537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:47.338722 2537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:47.338737 2537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:13:47.338752 2537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:13:47.340217 2537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:43409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:43409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:47.340467 2537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:47.340664 2537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:47.345162 2546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:47.345170 2544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:47.345252 2548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:47.345197 2537 server_base.cc:1034] running on GCE node
I20260808 02:13:47.345610 2537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:47.345842 2537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:47.346974 2537 hybrid_clock.cc:648] HybridClock initialized: now 1786155227346959 us; error 33 us; skew 500 ppm
Aug 08 02:13:47 dist-test-slave-1l3l krb5kdc[2522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:47.349376 2537 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:13:47.349817 2537 webserver.cc:460] Webserver started at http://127.19.58.254:42297/ using document root <none> and password file <none>
I20260808 02:13:47.350045 2537 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:47.350096 2537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:47.350217 2537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:47.351260 2537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "cf5050532e0d4879bb8e4759f39f3acd"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "9706d9f4fe5f8bb9e30101dcecd1bf00"
server_key_iv: "64c13ef67f7aad91d2304106321eb5d9"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.351590 2537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "cf5050532e0d4879bb8e4759f39f3acd"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "9706d9f4fe5f8bb9e30101dcecd1bf00"
server_key_iv: "64c13ef67f7aad91d2304106321eb5d9"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.352846 2537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:13:47.353513 2557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.353662 2537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:13:47.353745 2537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "cf5050532e0d4879bb8e4759f39f3acd"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "9706d9f4fe5f8bb9e30101dcecd1bf00"
server_key_iv: "64c13ef67f7aad91d2304106321eb5d9"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.353804 2537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:47.374099 2537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:47.376241 2537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:47.376381 2537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:47.380352 2537 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:43409
I20260808 02:13:47.380398 2628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:43409 every 8 connection(s)
I20260808 02:13:47.380725 2537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:13:47.381641 2630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:47.382747 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 2537
I20260808 02:13:47.382845 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:13:47.383042 19691 external_mini_cluster.cc:1428] Setting key bd2cf3ded475a193c92b2bf6c6fb952a
I20260808 02:13:47.384342 2630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd: Bootstrap starting.
I20260808 02:13:47.385062 2630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:47.385356 2630 log.cc:824] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:47.385979 2630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd: No bootstrap required, opened a new log
I20260808 02:13:47.388186 2630 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf5050532e0d4879bb8e4759f39f3acd" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43409 } }
I20260808 02:13:47.388357 2630 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:47.388420 2630 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf5050532e0d4879bb8e4759f39f3acd, State: Initialized, Role: FOLLOWER
I20260808 02:13:47.388549 2630 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf5050532e0d4879bb8e4759f39f3acd" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43409 } }
I20260808 02:13:47.388649 2630 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:13:47.388690 2630 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:13:47.388748 2630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:47.389361 2630 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf5050532e0d4879bb8e4759f39f3acd" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43409 } }
I20260808 02:13:47.389456 2630 leader_election.cc:302] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf5050532e0d4879bb8e4759f39f3acd; no voters:
I20260808 02:13:47.389597 2630 leader_election.cc:288] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:13:47.389638 2635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [term 1 FOLLOWER]: Leader election won for term 1
Aug 08 02:13:47 dist-test-slave-1l3l krb5kdc[2522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:47.389784 2635 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [term 1 LEADER]: Becoming Leader. State: Replica: cf5050532e0d4879bb8e4759f39f3acd, State: Running, Role: LEADER
I20260808 02:13:47.389856 2630 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:13:47.389922 2635 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf5050532e0d4879bb8e4759f39f3acd" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43409 } }
I20260808 02:13:47.390205 2635 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf5050532e0d4879bb8e4759f39f3acd. Latest consensus state: current_term: 1 leader_uuid: "cf5050532e0d4879bb8e4759f39f3acd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf5050532e0d4879bb8e4759f39f3acd" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43409 } } }
I20260808 02:13:47.390275 2635 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:47.390475 2642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:13:47.390959 2642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:13:47.392238 2636 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf5050532e0d4879bb8e4759f39f3acd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf5050532e0d4879bb8e4759f39f3acd" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43409 } } }
I20260808 02:13:47.392410 2636 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd [sys.catalog]: This master's current role is: LEADER
I20260808 02:13:47.392798 2642 catalog_manager.cc:1347] Generated new cluster ID: 3a00b97533624c538457cf167dec3c0e
I20260808 02:13:47.392838 2642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:13:47.393651 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.383557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.19.58.254:43409)
0808 02:13:47.383751 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.383760 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.383790 (+ 30us) server_negotiation.cc:407] Connection header received
0808 02:13:47.384202 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.384208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.384254 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.384363 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:47.385177 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.386147 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.387132 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.387515 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.387585 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.390192 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:47.390196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:47.390196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:47.390208 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:47.392040 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.392432 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.392435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.392436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.392539 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.392771 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.392773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.392773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.393073 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:47.393215 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.393325 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.393461 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260808 02:13:47.401379 2642 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:13:47.402038 2642 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:13:47.411748 2642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd: Generated new TSK 0
I20260808 02:13:47.411908 2642 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:13:47.446170 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43409
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:47.526929 2663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:47.527119 2663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:47.527140 2663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:47.528259 2663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:47.528299 2663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:47.528324 2663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:13:47.529724 2663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:47.529938 2663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:47.530169 2663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:47.534832 2671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:47.534844 2669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:47.535058 2668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:47.535032 2663 server_base.cc:1034] running on GCE node
I20260808 02:13:47.535226 2663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:47.535452 2663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:47.536592 2663 hybrid_clock.cc:648] HybridClock initialized: now 1786155227536581 us; error 31 us; skew 500 ppm
Aug 08 02:13:47 dist-test-slave-1l3l krb5kdc[2522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:47.538949 2663 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:13:47.539345 2663 webserver.cc:460] Webserver started at http://127.19.58.193:43741/ using document root <none> and password file <none>
I20260808 02:13:47.539535 2663 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:47.539575 2663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:47.539664 2663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:47.540683 2663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "50f447a33a394b869bd323345f720f11"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "5ac63b3db885e5592c9fd2b01a54d4d4"
server_key_iv: "c14eadded33645cdcd1f4e46e699ad37"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.540974 2663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "50f447a33a394b869bd323345f720f11"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "5ac63b3db885e5592c9fd2b01a54d4d4"
server_key_iv: "c14eadded33645cdcd1f4e46e699ad37"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.542104 2663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:47.542729 2686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.542867 2663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:47.542936 2663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "50f447a33a394b869bd323345f720f11"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "5ac63b3db885e5592c9fd2b01a54d4d4"
server_key_iv: "c14eadded33645cdcd1f4e46e699ad37"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.543000 2663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:47.551757 2663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:47.553714 2663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:47.553838 2663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:47.554028 2663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:47.554293 2663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:47.554327 2663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.554345 2663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:47.554368 2663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.560271 2663 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:46555
I20260808 02:13:47.560315 2852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:46555 every 8 connection(s)
I20260808 02:13:47.560654 2663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
Aug 08 02:13:47 dist-test-slave-1l3l krb5kdc[2522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:47.568746 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.561299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:44959 (local address 127.19.58.254:43409)
0808 02:13:47.561351 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.561353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.561533 (+ 180us) server_negotiation.cc:407] Connection header received
0808 02:13:47.562010 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.562011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.562020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.562072 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:47.562903 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.563399 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.564359 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.564563 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.564613 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.566467 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:47.566468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:47.566469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:47.566475 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:47.567733 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.568060 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.568061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.568061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.568108 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.568275 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.568275 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.568275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.568368 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:47.568419 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.568619 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.568683 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 02:13:47.568915 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.561256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:43409 (local address 127.19.58.193:44959)
0808 02:13:47.561420 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.561460 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.561932 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.562123 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.562126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.562654 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:47.562834 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.562839 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.563460 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.563461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.564293 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.564295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.564620 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.564620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.564713 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.565121 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:47.565129 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:47.566381 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:47.567800 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:47.567804 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:47.567806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:47.568008 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:47.568150 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:47.568151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:47.568151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:47.568234 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:47.568416 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:47.568419 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:47.568522 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.568598 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.568710 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260808 02:13:47.569415 2854 heartbeater.cc:347] Connected to a master server at 127.19.58.254:43409
I20260808 02:13:47.569499 2854 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:47.569635 2854 heartbeater.cc:511] Master 127.19.58.254:43409 requested a full tablet report, sending...
I20260808 02:13:47.570066 2579 ts_manager.cc:194] Registered new tserver with Master: 50f447a33a394b869bd323345f720f11 (127.19.58.193:46555)
I20260808 02:13:47.570686 2579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:44959
I20260808 02:13:47.570792 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 2663
I20260808 02:13:47.570904 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:13:47.571117 19691 external_mini_cluster.cc:1428] Setting key 70ec111792afcf7306b5f89a307efefe
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:13:47.596244 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43409
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:47.674858 2862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:47.675047 2862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:47.675069 2862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:47.676210 2862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:47.676251 2862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:47.676276 2862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:13:47.677667 2862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:47.677875 2862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:47.678087 2862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:47.682765 2869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:47.682765 2868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:47.682782 2872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:47.682778 2862 server_base.cc:1034] running on GCE node
I20260808 02:13:47.683146 2862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:47.683370 2862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:47.684509 2862 hybrid_clock.cc:648] HybridClock initialized: now 1786155227684487 us; error 38 us; skew 500 ppm
Aug 08 02:13:47 dist-test-slave-1l3l krb5kdc[2522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:47.686846 2862 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:13:47.687265 2862 webserver.cc:460] Webserver started at http://127.19.58.194:36195/ using document root <none> and password file <none>
I20260808 02:13:47.687481 2862 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:47.687529 2862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:47.687654 2862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:47.688687 2862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "502363d6744a44f78da56af1c7304bca"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "64ad3ca109e2822bd2446173721e8e68"
server_key_iv: "e44807caf2a75ecb0e9ca11446009b60"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.689015 2862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "502363d6744a44f78da56af1c7304bca"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "64ad3ca109e2822bd2446173721e8e68"
server_key_iv: "e44807caf2a75ecb0e9ca11446009b60"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.690281 2862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:47.691012 2883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.691148 2862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:47.691226 2862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "502363d6744a44f78da56af1c7304bca"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "64ad3ca109e2822bd2446173721e8e68"
server_key_iv: "e44807caf2a75ecb0e9ca11446009b60"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.691294 2862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:47.698074 2862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:47.700016 2862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:47.700140 2862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:47.700323 2862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:47.700601 2862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:47.700634 2862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.700652 2862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:47.700675 2862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.705893 2862 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:46217
I20260808 02:13:47.705952 3063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:46217 every 8 connection(s)
I20260808 02:13:47.706202 2862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:13:47.710685 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 2862
I20260808 02:13:47.710786 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:13:47.710991 19691 external_mini_cluster.cc:1428] Setting key 4e87168b23c8a801f86e4b595834a442
Aug 08 02:13:47 dist-test-slave-1l3l krb5kdc[2522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:47.714929 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.706878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:45271 (local address 127.19.58.254:43409)
0808 02:13:47.706920 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.706923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.707108 (+ 185us) server_negotiation.cc:407] Connection header received
0808 02:13:47.707519 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.707520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.707531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.707576 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:47.708446 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.708944 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.709827 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.710090 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.710139 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.712387 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:47.712388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:47.712389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:47.712396 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:47.713716 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.714057 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.714058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.714058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.714110 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.714357 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.714358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.714358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.714474 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:47.714531 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.714803 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.714863 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:13:47.715130 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.706857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:43409 (local address 127.19.58.194:45271)
0808 02:13:47.706995 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.707026 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.707447 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.707623 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.707626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.708197 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:47.708376 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.708382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.709007 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.709007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.709763 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.709764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.710155 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.710156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.710254 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.710700 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:47.710710 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:47.712303 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:47.713787 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:47.713793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:47.713795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:47.713990 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:47.714162 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:47.714162 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:47.714163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:47.714284 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:47.714535 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:47.714541 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:47.714681 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.714789 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.714932 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260808 02:13:47.715629 3064 heartbeater.cc:347] Connected to a master server at 127.19.58.254:43409
I20260808 02:13:47.715747 3064 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:47.715926 3064 heartbeater.cc:511] Master 127.19.58.254:43409 requested a full tablet report, sending...
I20260808 02:13:47.716281 2579 ts_manager.cc:194] Registered new tserver with Master: 502363d6744a44f78da56af1c7304bca (127.19.58.194:46217)
I20260808 02:13:47.716635 2579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:45271
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:13:47.736449 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43409
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:13:47.814536 3069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:13:47.814723 3069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:13:47.814751 3069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:13:47.815861 3069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:13:47.815905 3069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:13:47.815939 3069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:13:47.817301 3069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:13:47.817523 3069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:13:47.817725 3069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:13:47.822469 3076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:13:47.822470 3077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:47.822551 3069 server_base.cc:1034] running on GCE node
W20260808 02:13:47.822515 3079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:13:47.822830 3069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:13:47.823045 3069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:13:47.824188 3069 hybrid_clock.cc:648] HybridClock initialized: now 1786155227824173 us; error 24 us; skew 500 ppm
Aug 08 02:13:47 dist-test-slave-1l3l krb5kdc[2522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:13:47.826575 3069 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:13:47.826975 3069 webserver.cc:460] Webserver started at http://127.19.58.195:41171/ using document root <none> and password file <none>
I20260808 02:13:47.827178 3069 fs_manager.cc:359] Metadata directory not provided
I20260808 02:13:47.827225 3069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:13:47.827346 3069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:13:47.828351 3069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "f968735cfce147488987f78cfcacadd4"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "5ecfd827a82f127c16cb0983772b350f"
server_key_iv: "5586b44988766b688f9288c018484e13"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.828677 3069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "f968735cfce147488987f78cfcacadd4"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "5ecfd827a82f127c16cb0983772b350f"
server_key_iv: "5586b44988766b688f9288c018484e13"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.829856 3069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:13:47.830523 3090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.830653 3069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:13:47.830732 3069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "f968735cfce147488987f78cfcacadd4"
format_stamp: "Formatted at 2026-08-08 02:13:47 on dist-test-slave-1l3l"
server_key: "5ecfd827a82f127c16cb0983772b350f"
server_key_iv: "5586b44988766b688f9288c018484e13"
server_key_version: "encryptionkey@0"
I20260808 02:13:47.830787 3069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:13:47.846555 3069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:13:47.848438 3069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:13:47.848567 3069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:13:47.848764 3069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:13:47.849043 3069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:13:47.849078 3069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.849107 3069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:13:47.849123 3069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:13:47.855103 3069 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:37145
I20260808 02:13:47.855152 3251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:37145 every 8 connection(s)
I20260808 02:13:47.855464 3069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:13:47.860707 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 3069
I20260808 02:13:47.860793 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:13:47.860958 19691 external_mini_cluster.cc:1428] Setting key 74e5f20d820538563ce123a95d011f25
Aug 08 02:13:47 dist-test-slave-1l3l krb5kdc[2522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155227, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:13:47.864624 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.856219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:46317 (local address 127.19.58.254:43409)
0808 02:13:47.856265 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.856268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.856464 (+ 196us) server_negotiation.cc:407] Connection header received
0808 02:13:47.856919 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.856920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.856931 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.856994 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:47.857917 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.858435 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.859444 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.859651 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.859738 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.861865 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:47.861867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:47.861867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:47.861875 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:47.863163 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.863543 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.863544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.863544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.863598 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.863920 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.863921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.863921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.864061 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:47.864122 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.864447 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.864528 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:13:47.864833 3255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.856190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:43409 (local address 127.19.58.195:46317)
0808 02:13:47.856338 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.856387 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.856840 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.857062 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.857065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.857620 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:13:47.857808 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.857812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.858537 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.858538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.859348 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.859351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.859784 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.859785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.859913 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.860364 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:13:47.860373 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:13:47.861785 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:13:47.863233 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:47.863239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:47.863240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:47.863485 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:47.863669 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:13:47.863669 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:13:47.863670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:13:47.863831 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:13:47.864132 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:13:47.864139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:13:47.864297 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.864418 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.864588 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260808 02:13:47.865212 3252 heartbeater.cc:347] Connected to a master server at 127.19.58.254:43409
I20260808 02:13:47.865293 3252 heartbeater.cc:464] Registering TS with master...
I20260808 02:13:47.865458 3252 heartbeater.cc:511] Master 127.19.58.254:43409 requested a full tablet report, sending...
I20260808 02:13:47.865780 2579 ts_manager.cc:194] Registered new tserver with Master: f968735cfce147488987f78cfcacadd4 (127.19.58.195:37145)
I20260808 02:13:47.866137 2579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:46317
I20260808 02:13:47.872818 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:13:47.879462 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.874062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39844 (local address 127.19.58.254:43409)
0808 02:13:47.874122 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.874124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.874226 (+ 102us) server_negotiation.cc:407] Connection header received
0808 02:13:47.874282 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.874283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.874293 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.874334 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:13:47.874870 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.875346 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.876051 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.876236 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.876289 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.877375 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:13:47.877377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:13:47.877377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:13:47.877384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:13:47.878456 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.878746 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.878747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.878748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.878801 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:13:47.878962 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:13:47.878963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:13:47.878963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:13:47.879101 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:13:47.879185 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.879305 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.879375 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 02:13:47.880988 2579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:13:47.882499 2579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:13:47.888625 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.884371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:37145 (local address 127.0.0.1:56138)
0808 02:13:47.884531 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.884563 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.884634 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.884905 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.884907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.884910 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:47.884974 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.884976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.886267 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.886268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.887512 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.887515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.888375 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.888376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.888438 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.888442 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.888471 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.888543 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260808 02:13:47.888674 3278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.885010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.19.58.193:46555)
0808 02:13:47.885221 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.885226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.885235 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:47.885280 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.885282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.885298 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.885384 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:47.885574 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.886376 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.887494 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.888238 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.888317 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.888392 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.888539 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.888584 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260808 02:13:47.888633 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.884617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:46217 (local address 127.0.0.1:50564)
0808 02:13:47.884710 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.884719 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.884766 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.885476 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.885478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.885481 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:47.885525 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.885526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.886652 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.886653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.887763 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.887765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.888443 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.888443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.888496 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.888498 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.888546 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.888575 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":50,"threads_started":1}
I20260808 02:13:47.888878 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.884752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:46555 (local address 127.0.0.1:34688)
0808 02:13:47.884860 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:47.884868 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:47.884915 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:47.885466 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:47.885467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:47.885470 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:47.885516 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.885518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.886482 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.886483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.887421 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:47.887423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.888327 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.888328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:47.888406 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.888422 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 02:13:47.888485 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 02:13:47.888801 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":288,"thread_start_us":74,"threads_started":1}
I20260808 02:13:47.888670 3276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.884996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.19.58.194:46217)
0808 02:13:47.885310 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.885314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.885325 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:13:47.885361 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.885363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.885376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.885472 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:47.885606 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.886550 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.887831 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.888340 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.888437 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.888496 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.888555 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.888597 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":87,"threads_started":1}
I20260808 02:13:47.889061 3270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:47.884472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56138 (local address 127.19.58.195:37145)
0808 02:13:47.884637 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:47.884642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:47.884652 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:13:47.884723 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:47.884725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:47.884741 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:47.884813 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:47.885065 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.886161 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.887591 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:47.888286 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:47.888841 (+ 555us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:47.888918 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:47.888946 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 02:13:47.888986 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260808 02:13:47.889539 2747 tablet_service.cc:1467] Processing CreateTablet for tablet 1648d58f27754e1c979519b8dbec7ad9 (DEFAULT_TABLE table=test-table [id=297dee8bff69476e9b658106e583eae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:47.889539 2957 tablet_service.cc:1467] Processing CreateTablet for tablet 1648d58f27754e1c979519b8dbec7ad9 (DEFAULT_TABLE table=test-table [id=297dee8bff69476e9b658106e583eae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:47.889631 3164 tablet_service.cc:1467] Processing CreateTablet for tablet 1648d58f27754e1c979519b8dbec7ad9 (DEFAULT_TABLE table=test-table [id=297dee8bff69476e9b658106e583eae6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:13:47.889814 2747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1648d58f27754e1c979519b8dbec7ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:47.889881 3164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1648d58f27754e1c979519b8dbec7ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:47.889875 2957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1648d58f27754e1c979519b8dbec7ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:13:47.893023 3280 tablet_bootstrap.cc:492] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: Bootstrap starting.
I20260808 02:13:47.893023 3281 tablet_bootstrap.cc:492] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: Bootstrap starting.
I20260808 02:13:47.893487 3279 tablet_bootstrap.cc:492] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11: Bootstrap starting.
I20260808 02:13:47.893738 3280 tablet_bootstrap.cc:654] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:47.893738 3281 tablet_bootstrap.cc:654] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:47.894168 3279 tablet_bootstrap.cc:654] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11: Neither blocks nor log segments found. Creating new log.
I20260808 02:13:47.894337 3280 log.cc:824] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:47.894483 3281 log.cc:824] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:47.894719 3279 log.cc:824] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11: Log is configured to *not* fsync() on all Append() calls
I20260808 02:13:47.895190 3281 tablet_bootstrap.cc:492] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: No bootstrap required, opened a new log
I20260808 02:13:47.895200 3280 tablet_bootstrap.cc:492] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: No bootstrap required, opened a new log
I20260808 02:13:47.895256 3281 ts_tablet_manager.cc:1397] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:47.895256 3280 ts_tablet_manager.cc:1397] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:47.895403 3279 tablet_bootstrap.cc:492] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11: No bootstrap required, opened a new log
I20260808 02:13:47.895445 3279 ts_tablet_manager.cc:1397] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:13:47.896893 3279 raft_consensus.cc:348] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 } } peers { permanent_uuid: "502363d6744a44f78da56af1c7304bca" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46217 } } peers { permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 } }
I20260808 02:13:47.897020 3279 raft_consensus.cc:374] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:47.897051 3279 raft_consensus.cc:729] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f447a33a394b869bd323345f720f11, State: Initialized, Role: FOLLOWER
I20260808 02:13:47.897181 3279 consensus_queue.cc:260] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 } } peers { permanent_uuid: "502363d6744a44f78da56af1c7304bca" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46217 } } peers { permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 } }
I20260808 02:13:47.897163 3280 raft_consensus.cc:348] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 } } peers { permanent_uuid: "502363d6744a44f78da56af1c7304bca" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46217 } } peers { permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 } }
I20260808 02:13:47.897320 3280 raft_consensus.cc:374] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:47.897360 3280 raft_consensus.cc:729] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f968735cfce147488987f78cfcacadd4, State: Initialized, Role: FOLLOWER
I20260808 02:13:47.897285 3281 raft_consensus.cc:348] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 } } peers { permanent_uuid: "502363d6744a44f78da56af1c7304bca" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46217 } } peers { permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 } }
I20260808 02:13:47.897428 3281 raft_consensus.cc:374] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:13:47.897468 3281 raft_consensus.cc:729] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 502363d6744a44f78da56af1c7304bca, State: Initialized, Role: FOLLOWER
I20260808 02:13:47.897491 3279 ts_tablet_manager.cc:1428] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:47.897477 3280 consensus_queue.cc:260] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 } } peers { permanent_uuid: "502363d6744a44f78da56af1c7304bca" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46217 } } peers { permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 } }
I20260808 02:13:47.897616 2854 heartbeater.cc:503] Master 127.19.58.254:43409 was elected leader, sending a full tablet report...
I20260808 02:13:47.897606 3281 consensus_queue.cc:260] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 } } peers { permanent_uuid: "502363d6744a44f78da56af1c7304bca" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46217 } } peers { permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 } }
I20260808 02:13:47.897732 3280 ts_tablet_manager.cc:1428] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:13:47.897835 3252 heartbeater.cc:503] Master 127.19.58.254:43409 was elected leader, sending a full tablet report...
I20260808 02:13:47.898037 3064 heartbeater.cc:503] Master 127.19.58.254:43409 was elected leader, sending a full tablet report...
I20260808 02:13:47.898157 3281 ts_tablet_manager.cc:1428] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260808 02:13:47.956743 3065 tablet.cc:2369] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:13:48.006111 3287 raft_consensus.cc:482] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:13:48.006268 3287 raft_consensus.cc:504] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 } } peers { permanent_uuid: "502363d6744a44f78da56af1c7304bca" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46217 } } peers { permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 } }
I20260808 02:13:48.006640 3287 leader_election.cc:288] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f968735cfce147488987f78cfcacadd4 (127.19.58.195:37145), 50f447a33a394b869bd323345f720f11 (127.19.58.193:46555)
I20260808 02:13:48.010453 3278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:48.006981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:47421 (local address 127.19.58.193:46555)
0808 02:13:48.007042 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:48.007044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:48.007180 (+ 136us) server_negotiation.cc:407] Connection header received
0808 02:13:48.007232 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:48.007233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:48.007245 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:48.007299 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:48.007431 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:48.008133 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:48.009527 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:48.010105 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:48.010179 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:48.010187 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:48.010339 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 02:13:48.010379 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:13:48.010454 3270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:48.006908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:35609 (local address 127.19.58.195:37145)
0808 02:13:48.007002 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:48.007004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:48.007012 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:13:48.007039 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:48.007040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:48.007049 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:48.007098 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:13:48.007268 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:48.008051 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:48.009529 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:48.010105 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:48.010178 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:48.010187 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:48.010349 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 02:13:48.010391 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 02:13:48.010464 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:48.006902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:46555 (local address 127.19.58.194:47421)
0808 02:13:48.007095 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:48.007105 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:48.007175 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:48.007309 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:48.007310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:48.007311 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:48.007371 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:48.007372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:48.008213 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:48.008213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:48.009441 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:48.009443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:48.010180 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:48.010181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:48.010269 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:48.010273 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:13:48.010328 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 02:13:48.010377 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":39,"thread_start_us":133,"threads_started":1}
I20260808 02:13:48.010704 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:48.006801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:37145 (local address 127.19.58.194:35609)
0808 02:13:48.006850 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:48.006862 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:13:48.006919 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:13:48.007111 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:13:48.007112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:13:48.007114 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:13:48.007195 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:48.007197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:48.008145 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:48.008146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:48.009440 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:13:48.009443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:13:48.010180 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:13:48.010181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:13:48.010272 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:48.010274 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:13:48.010329 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 02:13:48.010645 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":18}
I20260808 02:13:48.010982 2779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1648d58f27754e1c979519b8dbec7ad9" candidate_uuid: "502363d6744a44f78da56af1c7304bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50f447a33a394b869bd323345f720f11" is_pre_election: true
I20260808 02:13:48.011040 3192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1648d58f27754e1c979519b8dbec7ad9" candidate_uuid: "502363d6744a44f78da56af1c7304bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f968735cfce147488987f78cfcacadd4" is_pre_election: true
I20260808 02:13:48.011153 2779 raft_consensus.cc:2454] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 502363d6744a44f78da56af1c7304bca in term 0.
I20260808 02:13:48.011196 3192 raft_consensus.cc:2454] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 502363d6744a44f78da56af1c7304bca in term 0.
I20260808 02:13:48.011380 2885 leader_election.cc:302] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 502363d6744a44f78da56af1c7304bca, 50f447a33a394b869bd323345f720f11; no voters:
I20260808 02:13:48.011502 3287 raft_consensus.cc:2781] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:13:48.011574 3287 raft_consensus.cc:482] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:13:48.011590 3287 raft_consensus.cc:3037] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:48.012348 3287 raft_consensus.cc:504] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 } } peers { permanent_uuid: "502363d6744a44f78da56af1c7304bca" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46217 } } peers { permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 } }
I20260808 02:13:48.012458 3287 leader_election.cc:288] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [CANDIDATE]: Term 1 election: Requested vote from peers f968735cfce147488987f78cfcacadd4 (127.19.58.195:37145), 50f447a33a394b869bd323345f720f11 (127.19.58.193:46555)
I20260808 02:13:48.012603 3192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1648d58f27754e1c979519b8dbec7ad9" candidate_uuid: "502363d6744a44f78da56af1c7304bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f968735cfce147488987f78cfcacadd4"
I20260808 02:13:48.012679 3192 raft_consensus.cc:3037] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:48.012684 2779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1648d58f27754e1c979519b8dbec7ad9" candidate_uuid: "502363d6744a44f78da56af1c7304bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50f447a33a394b869bd323345f720f11"
I20260808 02:13:48.012771 2779 raft_consensus.cc:3037] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:13:48.013403 3192 raft_consensus.cc:2454] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 502363d6744a44f78da56af1c7304bca in term 1.
I20260808 02:13:48.013517 2779 raft_consensus.cc:2454] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 502363d6744a44f78da56af1c7304bca in term 1.
I20260808 02:13:48.013559 2885 leader_election.cc:302] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 502363d6744a44f78da56af1c7304bca, f968735cfce147488987f78cfcacadd4; no voters:
I20260808 02:13:48.013677 3287 raft_consensus.cc:2781] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:13:48.013796 3287 raft_consensus.cc:686] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 1 LEADER]: Becoming Leader. State: Replica: 502363d6744a44f78da56af1c7304bca, State: Running, Role: LEADER
I20260808 02:13:48.013863 3287 consensus_queue.cc:237] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 } } peers { permanent_uuid: "502363d6744a44f78da56af1c7304bca" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46217 } } peers { permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 } }
I20260808 02:13:48.014704 2578 catalog_manager.cc:5495] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca reported cstate change: term changed from 0 to 1, leader changed from <none> to 502363d6744a44f78da56af1c7304bca (127.19.58.194). New cstate: current_term: 1 leader_uuid: "502363d6744a44f78da56af1c7304bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "502363d6744a44f78da56af1c7304bca" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 } health_report { overall_health: UNKNOWN } } }
I20260808 02:13:48.034991 3276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:48.032824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50576 (local address 127.19.58.194:46217)
0808 02:13:48.032873 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:13:48.032875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:13:48.032884 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:13:48.032935 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:13:48.032935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:13:48.032953 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:13:48.033012 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:13:48.033134 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:48.033621 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:48.034389 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:13:48.034576 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:13:48.034627 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:13:48.034726 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:13:48.034821 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:13:48.034875 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:13:48.034904 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 02:13:48.034920 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:13:48.036625 3192 raft_consensus.cc:1261] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4 [term 1 FOLLOWER]: Refusing update from remote peer 502363d6744a44f78da56af1c7304bca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:48.036649 2779 raft_consensus.cc:1261] T 1648d58f27754e1c979519b8dbec7ad9 P 50f447a33a394b869bd323345f720f11 [term 1 FOLLOWER]: Refusing update from remote peer 502363d6744a44f78da56af1c7304bca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:13:48.036986 3287 consensus_queue.cc:1035] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "50f447a33a394b869bd323345f720f11" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 46555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:48.037122 3291 consensus_queue.cc:1035] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f968735cfce147488987f78cfcacadd4" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:13:48.039305 3297 mvcc.cc:205] Tried to move back new op lower bound from 7316091814035595264 to 7316091813945315328. Current Snapshot: MvccSnapshot[applied={T|T < 7316091814035595264}]
I20260808 02:13:48.040097 3301 mvcc.cc:205] Tried to move back new op lower bound from 7316091814035595264 to 7316091813945315328. Current Snapshot: MvccSnapshot[applied={T|T < 7316091814035595264}]
I20260808 02:13:48.041371 2578 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:48.041463 2578 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:13:48.042237 2578 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf5050532e0d4879bb8e4759f39f3acd: Sending DeleteTablet for 3 replicas of tablet 1648d58f27754e1c979519b8dbec7ad9
I20260808 02:13:48.042522 2957 tablet_service.cc:1514] Processing DeleteTablet for tablet 1648d58f27754e1c979519b8dbec7ad9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:48 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:50564
I20260808 02:13:48.042562 2746 tablet_service.cc:1514] Processing DeleteTablet for tablet 1648d58f27754e1c979519b8dbec7ad9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:48 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:34688
I20260808 02:13:48.042644 3164 tablet_service.cc:1514] Processing DeleteTablet for tablet 1648d58f27754e1c979519b8dbec7ad9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:48 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:56138
I20260808 02:13:48.042739 2747 tablet_service.cc:1514] Processing DeleteTablet for tablet 1648d58f27754e1c979519b8dbec7ad9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:13:48 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:34688
I20260808 02:13:48.042853 3316 tablet_replica.cc:330] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: stopping tablet replica
I20260808 02:13:48.042884 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 2663
I20260808 02:13:48.043466 3316 raft_consensus.cc:2229] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:13:48.043577 3316 raft_consensus.cc:2258] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:48.047056 3316 ts_tablet_manager.cc:1905] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:48.048542 3316 ts_tablet_manager.cc:1918] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:48.048600 3316 log.cc:1196] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/1648d58f27754e1c979519b8dbec7ad9
I20260808 02:13:48.048857 3316 ts_tablet_manager.cc:1939] T 1648d58f27754e1c979519b8dbec7ad9 P f968735cfce147488987f78cfcacadd4: Deleting consensus metadata
I20260808 02:13:48.049340 2558 catalog_manager.cc:4841] TS f968735cfce147488987f78cfcacadd4 (127.19.58.195:37145): tablet 1648d58f27754e1c979519b8dbec7ad9 (table test-table [id=297dee8bff69476e9b658106e583eae6]) successfully deleted
I20260808 02:13:48.055176 3315 tablet_replica.cc:330] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: stopping tablet replica
I20260808 02:13:48.055272 3315 raft_consensus.cc:2229] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:13:48.055383 3315 raft_consensus.cc:2258] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:13:48.055847 3315 ts_tablet_manager.cc:1905] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:13:48.057575 3315 ts_tablet_manager.cc:1918] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:13:48.057691 3315 log.cc:1196] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/1648d58f27754e1c979519b8dbec7ad9
I20260808 02:13:48.058007 3315 ts_tablet_manager.cc:1939] T 1648d58f27754e1c979519b8dbec7ad9 P 502363d6744a44f78da56af1c7304bca: Deleting consensus metadata
I20260808 02:13:48.058454 2558 catalog_manager.cc:4841] TS 502363d6744a44f78da56af1c7304bca (127.19.58.194:46217): tablet 1648d58f27754e1c979519b8dbec7ad9 (table test-table [id=297dee8bff69476e9b658106e583eae6]) successfully deleted
W20260808 02:13:48.059257 2558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:46555 (error 108)
I20260808 02:13:48.059751 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:13:48.059594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:46555 (local address 127.0.0.1:34700)
0808 02:13:48.059648 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 02:13:48.059687 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 02:13:48.059835 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 2862
W20260808 02:13:48.063856 2558 catalog_manager.cc:4584] TS 50f447a33a394b869bd323345f720f11 (127.19.58.193:46555): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1648d58f27754e1c979519b8dbec7ad9: Network error: Client connection negotiation failed: client connection to 127.19.58.193:46555: connect: Connection refused (error 111)
I20260808 02:13:48.068056 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 3069
I20260808 02:13:48.075081 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 2537
2026-08-08T02:13:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2045 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:13:48 dist-test-slave-1l3l krb5kdc[3320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:13:48 dist-test-slave-1l3l krb5kdc[3320](info): set up 2 sockets
Aug 08 02:13:48 dist-test-slave-1l3l krb5kdc[3320](info): commencing operation
krb5kdc: starting...
W20260808 02:13:49.188975 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:13:49 dist-test-slave-1l3l krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:13:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:13:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.58.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.58.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/rangeradmin_127.19.58.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/rangeradmin_127.19.58.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.58.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.58.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/rangerlookup_127.19.58.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/rangerlookup_127.19.58.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.58.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.58.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/HTTP_127.19.58.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/HTTP_127.19.58.212@KRBTEST.COM.keytab.
I20260808 02:13:49.342980 19691 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDIPgti/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/postgres -l logfile start
2026-08-08 02:13:51.606 UTC [3369] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 02:13:51.606 UTC [3369] LOG: listening on IPv4 address "127.19.58.212", port 38983
2026-08-08 02:13:51.610 UTC [3369] LOG: listening on Unix socket "/tmp/.s.PGSQL.38983"
2026-08-08 02:13:51.630 UTC [3373] LOG: database system was shut down at 2026-08-08 02:13:50 UTC
2026-08-08 02:13:51.633 UTC [3369] LOG: database system is ready to accept connections
I20260808 02:13:52.661957 19691 mini_postgres.cc:96] Postgres bound to 38983
2026-08-08 02:13:52.668 UTC [3384] FATAL: database "slave" does not exist
127.19.58.212:38983 - accepting connections
I20260808 02:13:52.669265 19691 mini_ranger.cc:161] Starting Ranger...
I20260808 02:13:52.683497 19691 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 02:13:53.103960 19691 mini_ranger.cc:87] Created ranger Postgres database
I20260808 02:13:53.104031 19691 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-admin
2026-08-08 02:13:53,393 [I] DB FLAVOR :POSTGRES
2026-08-08 02:13:53,394 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:13:53,394 [I] Checking connection..
2026-08-08 02:13:53,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 02:13:53,650 [I] Checking connection passed.
2026-08-08 02:13:53,650 [I] --------- Verifying version history table ---------
2026-08-08 02:13:53,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:13:53,888 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:13:53,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:13:54,147 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:13:54,148 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 02:13:54,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 02:13:54.390 UTC [3485] WARNING: there is no transaction in progress
2026-08-08 02:13:54,398 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 02:13:54,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:13:54,657 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 02:13:54,657 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:13:54,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 02:13:54,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 02:13:55,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1l3l.c.gcp-upstream.internal','N') ;"
2026-08-08 02:13:55,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:13:55,636 [I] Table x_portal_user does not exist in database ranger
2026-08-08 02:13:55,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:13:55,895 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 02:13:55,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:13:56,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:13:56,384 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 02:13:56,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 02:13:57.288 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.303 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.318 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.332 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.347 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.364 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.425 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.431 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.444 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.453 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.462 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.475 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.484 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.489 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.495 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:57.501 UTC [3652] WARNING: there is no transaction in progress
2026-08-08 02:13:58,058 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 02:13:58,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 02:13:58,293 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 02:13:58,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:13:58,556 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 02:13:58,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:13:58,814 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 02:13:58,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:13:59,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:13:59,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1l3l.c.gcp-upstream.internal';"
2026-08-08 02:13:59,539 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 02:13:59,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 02:13:59,793 [I] DB_PATCHES have already been applied
I20260808 02:13:59.798413 19691 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDIPgti/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 02:13:59.798475 19691 mini_ranger.cc:193] Using host: 127.19.58.212
I20260808 02:13:59.802026 19691 mini_ranger.cc:227] Ranger admin URL: http://127.19.58.212:34405
Aug 08, 2026 2:14:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:14:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:14:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:14:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:14:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:14:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:14:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 02:14:00.959465 19691 mini_ranger.cc:160] Time spent starting Ranger: real 8.290s user 0.000s sys 0.010s
Aug 08, 2026 2:14:20 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:14:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 2:14:20 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:14:20 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:14:21 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 02:14:21.679838 19691 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.58.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.58.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/rangerkms_127.19.58.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/rangerkms_127.19.58.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.58.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.58.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.58.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/HTTP_127.19.58.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.58.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/HTTP_127.19.58.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 02:14:21 dist-test-slave-1l3l krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155261, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 02:14:21.758517 19691 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 02:14:21.767490 19691 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 02:14:22.518941 19691 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 02:14:22.519029 19691 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-kms
I20260808 02:14:22.519047 19691 mini_ranger_kms.cc:227] Using postgres at 127.19.58.212:38983
2026-08-08 02:14:22,750 [I] DB FLAVOR :POSTGRES
2026-08-08 02:14:22,750 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:14:22,750 [I] Checking connection
2026-08-08 02:14:22,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 02:14:22,983 [I] connection success
2026-08-08 02:14:22,984 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 02:14:22,984 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 02:14:22,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 02:14:23,232 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 02:14:23,232 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:14:23,232 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 02:14:23,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:38983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 02:14:23,499 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 02:14:23.958091 19691 mini_ranger_kms.cc:310] Created kms service
I20260808 02:14:24.129772 19691 mini_ranger_kms.cc:326] Created kudu user
I20260808 02:14:24.172986 19691 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 02:14:24 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155261, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:14:24.535444 19691 mini_ranger_kms.cc:384] Added ranger policy
I20260808 02:14:24.535564 19691 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 02:14:24.535588 19691 mini_ranger_kms.cc:240] Using host: 127.19.58.212
I20260808 02:14:24.540370 19691 mini_ranger_kms.cc:276] Ranger KMS PID: 3901
I20260808 02:14:24.540469 19691 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.58.212:60759
Aug 08, 2026 2:14:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:14:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:14:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:14:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:14:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:14:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 02:14:26 dist-test-slave-1l3l krb5kdc[3320](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155266, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.58.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:14:27.025187 19691 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.267s user 0.000s sys 0.215s
I20260808 02:14:27.025318 19691 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 02:14:27.025434 19691 mini_ranger_kms.cc:401] 127.19.58.212:60759/kms/v1/keys
Aug 08 02:14:27 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155266, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.58.212@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
Aug 08 02:14:27 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155266, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.58.212@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
Aug 08 02:14:29 dist-test-slave-1l3l krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:14:29.734164 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:36217
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:36217
--ranger_config_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:60759/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:14:29.813894 3972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:14:29.814106 3972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:14:29.814126 3972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:14:29.815241 3972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:14:29.815282 3972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:14:29.815310 3972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:14:29.815322 3972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:14:29.815333 3972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:14:29.816936 3972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.58.212:60759/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:36217
--ranger_config_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:36217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:14:29.817145 3972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:14:29.817373 3972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:14:29.823288 3982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:29.823351 3972 server_base.cc:1034] running on GCE node
W20260808 02:14:29.823388 3981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:29.823376 3985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:29.824616 3972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:14:29.824894 3972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:14:29.826066 3972 hybrid_clock.cc:648] HybridClock initialized: now 1786155269826045 us; error 39 us; skew 500 ppm
Aug 08 02:14:29 dist-test-slave-1l3l krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155269, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:14:29.831249 3972 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:14:29.831739 3972 webserver.cc:460] Webserver started at http://127.19.58.254:42275/ using document root <none> and password file <none>
I20260808 02:14:29.831943 3972 fs_manager.cc:359] Metadata directory not provided
I20260808 02:14:29.831984 3972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:14:29.832140 3972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:14:29 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155269, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:14:29.885998 3972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "50ccd28b844e449f840e40b0edb06e67"
format_stamp: "Formatted at 2026-08-08 02:14:29 on dist-test-slave-1l3l"
server_key: "bfd5f9b19a9aca44d7d5a05c2f10eb4f"
server_key_iv: "54823790e449cb407b61ab824326d15c"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:29.886462 3972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "50ccd28b844e449f840e40b0edb06e67"
format_stamp: "Formatted at 2026-08-08 02:14:29 on dist-test-slave-1l3l"
server_key: "bfd5f9b19a9aca44d7d5a05c2f10eb4f"
server_key_iv: "54823790e449cb407b61ab824326d15c"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:29.888386 3972 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 02:14:29.902544 3995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:29.902760 3972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:14:29.902851 3972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "50ccd28b844e449f840e40b0edb06e67"
format_stamp: "Formatted at 2026-08-08 02:14:29 on dist-test-slave-1l3l"
server_key: "bfd5f9b19a9aca44d7d5a05c2f10eb4f"
server_key_iv: "54823790e449cb407b61ab824326d15c"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:29.902947 3972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:14:29.913446 3972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:14:29.915236 3972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:14:29.915400 3972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:14:29.920233 3972 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:36217
I20260808 02:14:29.920262 4077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:36217 every 8 connection(s)
I20260808 02:14:29.920662 3972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:14:29.921334 4078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:14:29.923518 4078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67: Bootstrap starting.
I20260808 02:14:29.924103 4078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67: Neither blocks nor log segments found. Creating new log.
I20260808 02:14:29.924367 4078 log.cc:824] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67: Log is configured to *not* fsync() on all Append() calls
I20260808 02:14:29.925002 4078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67: No bootstrap required, opened a new log
I20260808 02:14:29.926988 4078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ccd28b844e449f840e40b0edb06e67" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 36217 } }
I20260808 02:14:29.927131 4078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:14:29.927163 4078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ccd28b844e449f840e40b0edb06e67, State: Initialized, Role: FOLLOWER
I20260808 02:14:29.927304 4078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ccd28b844e449f840e40b0edb06e67" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 36217 } }
I20260808 02:14:29.927378 4078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:14:29.927420 4078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:14:29.927467 4078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:14:29.928040 4078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ccd28b844e449f840e40b0edb06e67" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 36217 } }
I20260808 02:14:29.928138 4078 leader_election.cc:302] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50ccd28b844e449f840e40b0edb06e67; no voters:
I20260808 02:14:29.928305 4078 leader_election.cc:288] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:14:29.928345 4081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:14:29.928503 4081 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [term 1 LEADER]: Becoming Leader. State: Replica: 50ccd28b844e449f840e40b0edb06e67, State: Running, Role: LEADER
I20260808 02:14:29.928574 4078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:14:29.928618 4081 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ccd28b844e449f840e40b0edb06e67" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 36217 } }
I20260808 02:14:29.929108 4084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50ccd28b844e449f840e40b0edb06e67. Latest consensus state: current_term: 1 leader_uuid: "50ccd28b844e449f840e40b0edb06e67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ccd28b844e449f840e40b0edb06e67" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 36217 } } }
I20260808 02:14:29.929256 4084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [sys.catalog]: This master's current role is: LEADER
I20260808 02:14:29.929572 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 3972
I20260808 02:14:29.929670 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:14:29.929664 4082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50ccd28b844e449f840e40b0edb06e67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50ccd28b844e449f840e40b0edb06e67" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 36217 } } }
I20260808 02:14:29.929733 4082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67 [sys.catalog]: This master's current role is: LEADER
I20260808 02:14:29.929888 19691 external_mini_cluster.cc:1428] Setting key 95ffd39bb0b0e06efdff8a76053ac165
I20260808 02:14:29.930284 4078 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:14:29.930548 4097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 02:14:29 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:14:29.940902 4097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:14:29.945827 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:29.930391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40138 (local address 127.19.58.254:36217)
0808 02:14:29.930886 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:29.930896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:29.931197 (+ 301us) server_negotiation.cc:407] Connection header received
0808 02:14:29.933449 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:29.933495 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:29.933623 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:29.933766 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:14:29.934813 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:29.936329 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:29.937449 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:29.938103 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:29.938192 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:29.941296 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:14:29.941301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:14:29.941301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:14:29.941314 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:14:29.944035 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:29.944444 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:29.944447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:29.944448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:29.944566 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:29.944778 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:29.944779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:29.944779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:29.945092 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:14:29.945229 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:29.945477 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 02:14:29.945604 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":72,"threads_started":1}
I20260808 02:14:29.948135 4097 catalog_manager.cc:1347] Generated new cluster ID: df8d415dbb354be582d21b65923e8d40
I20260808 02:14:29.948192 4097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:14:29.964141 4097 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:14:29.964919 4097 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:14:29.982107 4097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67: Generated new TSK 0
I20260808 02:14:29.982293 4097 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:14:30.586774 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:60759/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:36217
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:14:30.696105 4140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:14:30.696336 4140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:14:30.696368 4140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:14:30.697502 4140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:14:30.697541 4140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:14:30.697575 4140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:14:30.699024 4140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.58.212:60759/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:36217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:14:30.699236 4140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:14:30.699436 4140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:14:30.704195 4147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:30.704217 4146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:30.704476 4149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:30.704571 4140 server_base.cc:1034] running on GCE node
I20260808 02:14:30.704749 4140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:14:30.704934 4140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:14:30.706067 4140 hybrid_clock.cc:648] HybridClock initialized: now 1786155270706055 us; error 33 us; skew 500 ppm
Aug 08 02:14:30 dist-test-slave-1l3l krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:14:30.708756 4140 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:14:30.709306 4140 webserver.cc:460] Webserver started at http://127.19.58.193:41803/ using document root <none> and password file <none>
I20260808 02:14:30.709563 4140 fs_manager.cc:359] Metadata directory not provided
I20260808 02:14:30.709633 4140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:14:30.709779 4140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:14:30 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:14:30.723834 4140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "ea982a7ef94d4fd99c2058403afbbebf"
format_stamp: "Formatted at 2026-08-08 02:14:30 on dist-test-slave-1l3l"
server_key: "2d102f337c19de26918c20a169a9ce04"
server_key_iv: "676b97cadec61ade3350d1bdad4ef5bf"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:30.724472 4140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "ea982a7ef94d4fd99c2058403afbbebf"
format_stamp: "Formatted at 2026-08-08 02:14:30 on dist-test-slave-1l3l"
server_key: "2d102f337c19de26918c20a169a9ce04"
server_key_iv: "676b97cadec61ade3350d1bdad4ef5bf"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:30.726388 4140 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:14:30.743438 4160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:30.743772 4140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:14:30.743880 4140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "ea982a7ef94d4fd99c2058403afbbebf"
format_stamp: "Formatted at 2026-08-08 02:14:30 on dist-test-slave-1l3l"
server_key: "2d102f337c19de26918c20a169a9ce04"
server_key_iv: "676b97cadec61ade3350d1bdad4ef5bf"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:30.743984 4140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:14:30.767109 4140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:14:30.769675 4140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:14:30.769894 4140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:14:30.770147 4140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:14:30.770586 4140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:14:30.770689 4140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:30.770787 4140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:14:30.770859 4140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:30.779418 4140 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:35373
I20260808 02:14:30.779886 4140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:14:30.782361 4333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:35373 every 8 connection(s)
I20260808 02:14:30.785832 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 4140
I20260808 02:14:30.785923 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:14:30.786098 19691 external_mini_cluster.cc:1428] Setting key 073a05195633f40cbba60a8b4383e42e
Aug 08 02:14:30 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:14:30.794199 4338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:30.782955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:36217 (local address 127.19.58.193:47101)
0808 02:14:30.783225 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:30.783278 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:30.784025 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:30.784452 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:30.784456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:30.784697 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:14:30.784980 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:30.784987 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:30.785801 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:30.785803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:30.787069 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:30.787072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:30.787530 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:30.787531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:30.787673 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:30.788250 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:14:30.788260 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:14:30.789921 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:14:30.792397 (+ 2476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:30.792405 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:30.792407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:30.792706 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:30.792954 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:30.792954 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:30.792955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:30.793092 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:30.793437 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:14:30.793445 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:14:30.793612 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 02:14:30.793727 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 02:14:30.793906 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":141,"threads_started":1}
I20260808 02:14:30.794698 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:30.783410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:47101 (local address 127.19.58.254:36217)
0808 02:14:30.784220 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:30.784222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:30.784230 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:14:30.784257 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:30.784258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:30.784270 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:30.784337 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:14:30.785108 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:30.785698 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:30.787168 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:30.787436 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:30.787506 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:30.790034 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:14:30.790036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:14:30.790036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:14:30.790045 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:14:30.792301 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:30.792793 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:30.792795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:30.792795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:30.792876 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:30.793191 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:30.793192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:30.793193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:30.793340 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:14:30.793407 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:30.794513 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0808 02:14:30.794603 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":191,"threads_started":1}
I20260808 02:14:30.795174 4334 heartbeater.cc:347] Connected to a master server at 127.19.58.254:36217
I20260808 02:14:30.795334 4334 heartbeater.cc:464] Registering TS with master...
I20260808 02:14:30.795584 4334 heartbeater.cc:511] Master 127.19.58.254:36217 requested a full tablet report, sending...
I20260808 02:14:30.796219 4029 ts_manager.cc:194] Registered new tserver with Master: ea982a7ef94d4fd99c2058403afbbebf (127.19.58.193:35373)
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
I20260808 02:14:30.797290 4029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:47101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:14:30.816406 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:60759/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:36217
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:14:30.929116 4346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:14:30.929435 4346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:14:30.929481 4346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:14:30.931159 4346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:14:30.931227 4346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:14:30.931267 4346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:14:30.933516 4346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.58.212:60759/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:36217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:14:30.933755 4346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:14:30.934091 4346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:14:30.940618 4359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:30.940661 4355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:30.940982 4356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:30.941148 4346 server_base.cc:1034] running on GCE node
I20260808 02:14:30.941336 4346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:14:30.941577 4346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:14:30.942731 4346 hybrid_clock.cc:648] HybridClock initialized: now 1786155270942700 us; error 44 us; skew 500 ppm
Aug 08 02:14:30 dist-test-slave-1l3l krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:14:30.946260 4346 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:14:30.946786 4346 webserver.cc:460] Webserver started at http://127.19.58.194:34683/ using document root <none> and password file <none>
I20260808 02:14:30.947045 4346 fs_manager.cc:359] Metadata directory not provided
I20260808 02:14:30.947113 4346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:14:30.947317 4346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:14:30 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
Aug 08 02:14:30 dist-test-slave-1l3l krb5kdc[3320](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:14:30.964458 4346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "cdd5420345714178a9d713c6516e787d"
format_stamp: "Formatted at 2026-08-08 02:14:30 on dist-test-slave-1l3l"
server_key: "7759dc06eb604e5575df90f343dfeda4"
server_key_iv: "8e60239c2b2417c82221260f33c3fce9"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:30.964929 4346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "cdd5420345714178a9d713c6516e787d"
format_stamp: "Formatted at 2026-08-08 02:14:30 on dist-test-slave-1l3l"
server_key: "7759dc06eb604e5575df90f343dfeda4"
server_key_iv: "8e60239c2b2417c82221260f33c3fce9"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:30.966845 4346 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:14:30.989373 4368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:30.989609 4346 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260808 02:14:30.989710 4346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "cdd5420345714178a9d713c6516e787d"
format_stamp: "Formatted at 2026-08-08 02:14:30 on dist-test-slave-1l3l"
server_key: "7759dc06eb604e5575df90f343dfeda4"
server_key_iv: "8e60239c2b2417c82221260f33c3fce9"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:30.989809 4346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:14:31.018036 4346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:14:31.020587 4346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:14:31.020803 4346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:14:31.021087 4346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:14:31.021510 4346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:14:31.021603 4346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:31.021675 4346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:14:31.021733 4346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:31.030505 4346 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:35739
I20260808 02:14:31.031006 4521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:35739 every 8 connection(s)
I20260808 02:14:31.031016 4346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:14:31.035405 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 4346
I20260808 02:14:31.035518 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:14:31.035745 19691 external_mini_cluster.cc:1428] Setting key 5d73f62cc14a647f5ff5bad969f5c78e
Aug 08 02:14:31 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:14:31.046803 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.033027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:53643 (local address 127.19.58.254:36217)
0808 02:14:31.033093 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:31.033095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:31.033587 (+ 492us) server_negotiation.cc:407] Connection header received
0808 02:14:31.035517 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:31.035519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:31.035535 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:31.035566 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:14:31.036431 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.037088 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.040342 (+ 3254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.040661 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.040706 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.043423 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:14:31.043425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:14:31.043425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:14:31.043434 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:14:31.045179 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:31.045656 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:31.045657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:31.045657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:31.045715 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:31.045990 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:31.045991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:31.045991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:31.046131 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:14:31.046174 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:31.046602 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 02:14:31.046699 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:14:31.047785 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.033107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:36217 (local address 127.19.58.194:53643)
0808 02:14:31.033442 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:31.033499 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:31.035367 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:31.035785 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:31.035789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:31.036036 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:14:31.036328 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.036335 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.037208 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.037209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.040235 (+ 3026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.040239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.040798 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.040799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.040944 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.041562 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:14:31.041573 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:14:31.043303 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:14:31.045276 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:31.045283 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:31.045285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:31.045566 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:31.045780 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:31.045781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:31.045781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:31.045906 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:31.046220 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:14:31.046226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:14:31.046434 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 02:14:31.047343 (+ 909us) client_negotiation.cc:240] Negotiation successful
0808 02:14:31.047515 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":100,"threads_started":1}
I20260808 02:14:31.048520 4522 heartbeater.cc:347] Connected to a master server at 127.19.58.254:36217
I20260808 02:14:31.048614 4522 heartbeater.cc:464] Registering TS with master...
I20260808 02:14:31.048800 4522 heartbeater.cc:511] Master 127.19.58.254:36217 requested a full tablet report, sending...
I20260808 02:14:31.049259 4029 ts_manager.cc:194] Registered new tserver with Master: cdd5420345714178a9d713c6516e787d (127.19.58.194:35739)
I20260808 02:14:31.049772 4029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:53643
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:14:31.072443 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:60759/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:36217
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:14:31.194427 4531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:14:31.194710 4531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:14:31.194751 4531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:14:31.196483 4531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:14:31.196548 4531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:14:31.196595 4531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:14:31.198755 4531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.58.212:60759/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:36217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:14:31.198989 4531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:14:31.199321 4531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:14:31.205199 4538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:31.205284 4542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:14:31.205431 4536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:14:31.205874 4531 server_base.cc:1034] running on GCE node
I20260808 02:14:31.206048 4531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:14:31.206274 4531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:14:31.207433 4531 hybrid_clock.cc:648] HybridClock initialized: now 1786155271207411 us; error 41 us; skew 500 ppm
Aug 08 02:14:31 dist-test-slave-1l3l krb5kdc[3320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155271, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:14:31.209995 4531 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:14:31.210441 4531 webserver.cc:460] Webserver started at http://127.19.58.195:42335/ using document root <none> and password file <none>
I20260808 02:14:31.210649 4531 fs_manager.cc:359] Metadata directory not provided
I20260808 02:14:31.210697 4531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:14:31.210830 4531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:14:31 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155271, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:14:31.224236 4531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "6284d745db0d4f7c943703ad6eb3413c"
format_stamp: "Formatted at 2026-08-08 02:14:31 on dist-test-slave-1l3l"
server_key: "aa672ffba746b9f560d7c652cc191b00"
server_key_iv: "ecb349e6c7ad97c4e66a3f88cf9cb0ba"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:31.224736 4531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "6284d745db0d4f7c943703ad6eb3413c"
format_stamp: "Formatted at 2026-08-08 02:14:31 on dist-test-slave-1l3l"
server_key: "aa672ffba746b9f560d7c652cc191b00"
server_key_iv: "ecb349e6c7ad97c4e66a3f88cf9cb0ba"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:31.232312 4531 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.000s
I20260808 02:14:31.253604 4550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:31.253840 4531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:14:31.253940 4531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "6284d745db0d4f7c943703ad6eb3413c"
format_stamp: "Formatted at 2026-08-08 02:14:31 on dist-test-slave-1l3l"
server_key: "aa672ffba746b9f560d7c652cc191b00"
server_key_iv: "ecb349e6c7ad97c4e66a3f88cf9cb0ba"
server_key_version: "kuduclusterkey@0"
I20260808 02:14:31.254035 4531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:14:31.277719 4531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:14:31.279734 4531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:14:31.279896 4531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:14:31.280141 4531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:14:31.280512 4531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:14:31.280551 4531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:31.280581 4531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:14:31.280634 4531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:14:31.294358 4531 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:46553
I20260808 02:14:31.294883 4531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:14:31.299851 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 4531
I20260808 02:14:31.299970 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:14:31.300207 19691 external_mini_cluster.cc:1428] Setting key 804d05d18d6c93df4afdec78e633312a
I20260808 02:14:31.307245 4707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:46553 every 8 connection(s)
Aug 08 02:14:31 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155271, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:14:31.317649 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.307804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:49761 (local address 127.19.58.254:36217)
0808 02:14:31.307853 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:31.307855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:31.308129 (+ 274us) server_negotiation.cc:407] Connection header received
0808 02:14:31.308624 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:31.308625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:31.308637 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:31.308704 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:14:31.309286 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.309801 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.310810 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.311124 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.311197 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.314025 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:14:31.314026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:14:31.314026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:14:31.314035 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:14:31.315977 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:31.316497 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:31.316498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:31.316499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:31.316562 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:31.316873 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:31.316874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:31.316875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:31.317012 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:14:31.317057 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:31.317460 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 02:14:31.317553 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 02:14:31.318415 4711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.307479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:36217 (local address 127.19.58.195:49761)
0808 02:14:31.307990 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:31.308039 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:31.308530 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:31.308779 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:31.308784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:31.309000 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:14:31.309197 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.309201 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.309884 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.309885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.310722 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.310725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.311281 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.311282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.311446 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.312123 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:14:31.312133 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:14:31.313873 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:14:31.316076 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:31.316085 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:31.316087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:31.316399 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:31.316648 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:14:31.316649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:14:31.316649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:14:31.316785 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:14:31.317108 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:14:31.317115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:14:31.317279 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 02:14:31.317993 (+ 714us) client_negotiation.cc:240] Negotiation successful
0808 02:14:31.318170 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":107,"threads_started":1}
I20260808 02:14:31.319063 4709 heartbeater.cc:347] Connected to a master server at 127.19.58.254:36217
I20260808 02:14:31.319154 4709 heartbeater.cc:464] Registering TS with master...
I20260808 02:14:31.319337 4709 heartbeater.cc:511] Master 127.19.58.254:36217 requested a full tablet report, sending...
I20260808 02:14:31.319795 4029 ts_manager.cc:194] Registered new tserver with Master: 6284d745db0d4f7c943703ad6eb3413c (127.19.58.195:46553)
I20260808 02:14:31.320312 4029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:49761
I20260808 02:14:31.322757 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:14:31.332832 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.324601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.19.58.254:36217)
0808 02:14:31.324648 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:31.324649 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:31.324876 (+ 227us) server_negotiation.cc:407] Connection header received
0808 02:14:31.324989 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:31.324990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:31.325000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:31.325028 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:14:31.325794 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.326568 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.327660 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.327967 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.328009 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.329639 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:14:31.329641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:14:31.329641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:14:31.329648 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:14:31.331538 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:31.331994 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:31.331995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:31.331995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:31.332053 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:14:31.332264 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:14:31.332265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:14:31.332266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:14:31.332408 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:14:31.332516 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:31.332653 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 02:14:31.332730 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:14:31.334736 4029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:14:31.337142 4029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:14:31.348358 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.340188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:46553 (local address 127.0.0.1:47420)
0808 02:14:31.340609 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:31.340651 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:31.340719 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:31.343876 (+ 3157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:31.343877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:31.343879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:14:31.343920 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.343922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.344857 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.344858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.346111 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.346114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.348058 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.348059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.348143 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.348157 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 02:14:31.348226 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:14:31.348280 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":91,"threads_started":1}
I20260808 02:14:31.349399 4727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.340475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47420 (local address 127.19.58.195:46553)
0808 02:14:31.343179 (+ 2704us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:31.343184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:31.343193 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:14:31.343230 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:31.343232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:31.343247 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:31.343999 (+ 752us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:14:31.344100 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.344772 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.347119 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.347939 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.348010 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.348089 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:31.348866 (+ 777us) server_negotiation.cc:299] Negotiation successful
0808 02:14:31.348911 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2665,"thread_start_us":93,"threads_started":1}
I20260808 02:14:31.350219 4599 tablet_service.cc:1467] Processing CreateTablet for tablet d4d1687d95724856bfaa2cffd6c24ed9 (DEFAULT_TABLE table=test-table [id=51489002f649424da5dc2ba8e5ab9764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:14:31.350589 4599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4d1687d95724856bfaa2cffd6c24ed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:14:31.351100 4724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.340216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:35373 (local address 127.0.0.1:48638)
0808 02:14:31.342764 (+ 2548us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:31.342775 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:31.342841 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:31.343371 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:31.343374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:31.343377 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:14:31.343446 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.343448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.345901 (+ 2453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.345902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.346999 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.347003 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.350876 (+ 3873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.350877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.350939 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.350943 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:14:31.350990 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 02:14:31.351033 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2528,"thread_start_us":92,"threads_started":1}
I20260808 02:14:31.351413 4729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.340774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48638 (local address 127.19.58.193:35373)
0808 02:14:31.341069 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:31.341074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:31.342902 (+ 1828us) server_negotiation.cc:407] Connection header received
0808 02:14:31.342945 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:31.342947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:31.342965 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:31.343040 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:14:31.343543 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.345804 (+ 2261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.348230 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.350724 (+ 2494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.350786 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.350840 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:31.351287 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 02:14:31.351344 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":80,"threads_started":1}
I20260808 02:14:31.352015 4226 tablet_service.cc:1467] Processing CreateTablet for tablet d4d1687d95724856bfaa2cffd6c24ed9 (DEFAULT_TABLE table=test-table [id=51489002f649424da5dc2ba8e5ab9764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:14:31.352391 4226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4d1687d95724856bfaa2cffd6c24ed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:14:31.355822 4731 tablet_bootstrap.cc:492] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Bootstrap starting.
I20260808 02:14:31.356878 4731 tablet_bootstrap.cc:654] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Neither blocks nor log segments found. Creating new log.
I20260808 02:14:31.357239 4731 log.cc:824] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:14:31.358121 4731 tablet_bootstrap.cc:492] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: No bootstrap required, opened a new log
I20260808 02:14:31.358197 4731 ts_tablet_manager.cc:1397] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:14:31.358630 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.340449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35739 (local address 127.0.0.1:57484)
0808 02:14:31.348735 (+ 8286us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:31.348744 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:31.348780 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:31.349254 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:31.349255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:31.349257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:14:31.349293 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.349295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.356282 (+ 6987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.356283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.357369 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.357372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.358426 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.358427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.358491 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.358494 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:14:31.358522 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 02:14:31.358562 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8275,"thread_start_us":85,"threads_started":1}
I20260808 02:14:31.359021 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.340953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57484 (local address 127.19.58.194:35739)
0808 02:14:31.348190 (+ 7237us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:31.348195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:31.349090 (+ 895us) server_negotiation.cc:407] Connection header received
0808 02:14:31.349140 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:31.349142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:31.349158 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:31.349379 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:14:31.349463 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.350445 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.357500 (+ 7055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.358329 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.358789 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.358877 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:31.358905 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 02:14:31.358944 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7193,"thread_start_us":96,"threads_started":1}
I20260808 02:14:31.359616 4436 tablet_service.cc:1467] Processing CreateTablet for tablet d4d1687d95724856bfaa2cffd6c24ed9 (DEFAULT_TABLE table=test-table [id=51489002f649424da5dc2ba8e5ab9764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:14:31.359988 4436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4d1687d95724856bfaa2cffd6c24ed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:14:31.360117 4731 raft_consensus.cc:348] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 } } peers { permanent_uuid: "ea982a7ef94d4fd99c2058403afbbebf" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35373 } } peers { permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 } }
I20260808 02:14:31.360265 4731 raft_consensus.cc:374] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:14:31.360294 4731 raft_consensus.cc:729] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6284d745db0d4f7c943703ad6eb3413c, State: Initialized, Role: FOLLOWER
I20260808 02:14:31.360407 4731 consensus_queue.cc:260] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 } } peers { permanent_uuid: "ea982a7ef94d4fd99c2058403afbbebf" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35373 } } peers { permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 } }
I20260808 02:14:31.360652 4731 ts_tablet_manager.cc:1428] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:14:31.363811 4709 heartbeater.cc:503] Master 127.19.58.254:36217 was elected leader, sending a full tablet report...
I20260808 02:14:31.363960 4733 tablet_bootstrap.cc:492] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf: Bootstrap starting.
I20260808 02:14:31.365038 4733 tablet_bootstrap.cc:654] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf: Neither blocks nor log segments found. Creating new log.
I20260808 02:14:31.365368 4733 log.cc:824] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf: Log is configured to *not* fsync() on all Append() calls
I20260808 02:14:31.365686 4736 tablet_bootstrap.cc:492] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Bootstrap starting.
I20260808 02:14:31.366288 4733 tablet_bootstrap.cc:492] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf: No bootstrap required, opened a new log
I20260808 02:14:31.366438 4733 ts_tablet_manager.cc:1397] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:14:31.366650 4736 tablet_bootstrap.cc:654] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Neither blocks nor log segments found. Creating new log.
I20260808 02:14:31.367012 4736 log.cc:824] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:14:31.370751 4733 raft_consensus.cc:348] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 } } peers { permanent_uuid: "ea982a7ef94d4fd99c2058403afbbebf" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35373 } } peers { permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 } }
I20260808 02:14:31.371001 4733 raft_consensus.cc:374] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:14:31.371084 4733 raft_consensus.cc:729] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea982a7ef94d4fd99c2058403afbbebf, State: Initialized, Role: FOLLOWER
I20260808 02:14:31.371301 4733 consensus_queue.cc:260] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 } } peers { permanent_uuid: "ea982a7ef94d4fd99c2058403afbbebf" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35373 } } peers { permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 } }
I20260808 02:14:31.371596 4733 ts_tablet_manager.cc:1428] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260808 02:14:31.371740 4334 heartbeater.cc:503] Master 127.19.58.254:36217 was elected leader, sending a full tablet report...
I20260808 02:14:31.373039 4736 tablet_bootstrap.cc:492] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: No bootstrap required, opened a new log
I20260808 02:14:31.373605 4736 ts_tablet_manager.cc:1397] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 02:14:31.375648 4736 raft_consensus.cc:348] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 } } peers { permanent_uuid: "ea982a7ef94d4fd99c2058403afbbebf" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35373 } } peers { permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 } }
I20260808 02:14:31.375905 4736 raft_consensus.cc:374] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:14:31.375996 4736 raft_consensus.cc:729] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd5420345714178a9d713c6516e787d, State: Initialized, Role: FOLLOWER
I20260808 02:14:31.376255 4736 consensus_queue.cc:260] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 } } peers { permanent_uuid: "ea982a7ef94d4fd99c2058403afbbebf" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35373 } } peers { permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 } }
I20260808 02:14:31.376672 4736 ts_tablet_manager.cc:1428] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:14:31.376842 4522 heartbeater.cc:503] Master 127.19.58.254:36217 was elected leader, sending a full tablet report...
I20260808 02:14:31.416754 4741 raft_consensus.cc:482] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:14:31.416913 4741 raft_consensus.cc:504] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 } } peers { permanent_uuid: "ea982a7ef94d4fd99c2058403afbbebf" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35373 } } peers { permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 } }
I20260808 02:14:31.417292 4741 leader_election.cc:288] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6284d745db0d4f7c943703ad6eb3413c (127.19.58.195:46553), cdd5420345714178a9d713c6516e787d (127.19.58.194:35739)
I20260808 02:14:31.426605 4727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.417886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:41809 (local address 127.19.58.195:46553)
0808 02:14:31.417932 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:31.417934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:31.422700 (+ 4766us) server_negotiation.cc:407] Connection header received
0808 02:14:31.422824 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:31.422826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:31.422836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:31.422866 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:14:31.423161 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.423958 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.425640 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.426215 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.426263 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.426273 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:31.426489 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 02:14:31.426530 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:14:31.427060 4744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.417552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:46553 (local address 127.19.58.193:41809)
0808 02:14:31.419392 (+ 1840us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:31.419405 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:31.422738 (+ 3333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:31.422937 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:31.422938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:31.422953 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:14:31.423071 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.423073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.424108 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.424109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.425376 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.425379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.426336 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.426336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.426414 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.426417 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:14:31.426921 (+ 504us) client_negotiation.cc:240] Negotiation successful
0808 02:14:31.426969 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1817,"thread_start_us":116,"threads_started":1}
I20260808 02:14:31.427564 4647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d1687d95724856bfaa2cffd6c24ed9" candidate_uuid: "ea982a7ef94d4fd99c2058403afbbebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6284d745db0d4f7c943703ad6eb3413c" is_pre_election: true
I20260808 02:14:31.427749 4647 raft_consensus.cc:2454] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea982a7ef94d4fd99c2058403afbbebf in term 0.
I20260808 02:14:31.429543 4162 leader_election.cc:302] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6284d745db0d4f7c943703ad6eb3413c, ea982a7ef94d4fd99c2058403afbbebf; no voters:
I20260808 02:14:31.429684 4741 raft_consensus.cc:2781] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:14:31.429721 4741 raft_consensus.cc:482] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:14:31.429741 4741 raft_consensus.cc:3037] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:14:31.430758 4741 raft_consensus.cc:504] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 } } peers { permanent_uuid: "ea982a7ef94d4fd99c2058403afbbebf" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35373 } } peers { permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 } }
I20260808 02:14:31.430905 4741 leader_election.cc:288] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [CANDIDATE]: Term 1 election: Requested vote from peers 6284d745db0d4f7c943703ad6eb3413c (127.19.58.195:46553), cdd5420345714178a9d713c6516e787d (127.19.58.194:35739)
I20260808 02:14:31.431246 4647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d1687d95724856bfaa2cffd6c24ed9" candidate_uuid: "ea982a7ef94d4fd99c2058403afbbebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6284d745db0d4f7c943703ad6eb3413c"
I20260808 02:14:31.431325 4647 raft_consensus.cc:3037] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:14:31.432171 4647 raft_consensus.cc:2454] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea982a7ef94d4fd99c2058403afbbebf in term 1.
I20260808 02:14:31.432405 4162 leader_election.cc:302] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6284d745db0d4f7c943703ad6eb3413c, ea982a7ef94d4fd99c2058403afbbebf; no voters:
I20260808 02:14:31.432524 4741 raft_consensus.cc:2781] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:14:31.432683 4741 raft_consensus.cc:686] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [term 1 LEADER]: Becoming Leader. State: Replica: ea982a7ef94d4fd99c2058403afbbebf, State: Running, Role: LEADER
I20260808 02:14:31.432770 4741 consensus_queue.cc:237] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 } } peers { permanent_uuid: "ea982a7ef94d4fd99c2058403afbbebf" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35373 } } peers { permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 } }
I20260808 02:14:31.433616 4029 catalog_manager.cc:5495] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf reported cstate change: term changed from 0 to 1, leader changed from <none> to ea982a7ef94d4fd99c2058403afbbebf (127.19.58.193). New cstate: current_term: 1 leader_uuid: "ea982a7ef94d4fd99c2058403afbbebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea982a7ef94d4fd99c2058403afbbebf" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 35373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 } health_report { overall_health: UNKNOWN } } }
I20260808 02:14:31.435284 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.418250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:57481 (local address 127.19.58.194:35739)
0808 02:14:31.423773 (+ 5523us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:31.423775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:31.431872 (+ 8097us) server_negotiation.cc:407] Connection header received
0808 02:14:31.431973 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:31.431974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:31.431984 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:31.432014 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:14:31.432216 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.432940 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.434357 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.434895 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.434947 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.434957 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:31.435174 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 02:14:31.435218 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5487}
Aug 08 02:14:31 dist-test-slave-1l3l krb5kdc[3320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155270, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:14:31.435753 4743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.418054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35739 (local address 127.19.58.193:57481)
0808 02:14:31.431792 (+ 13738us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:31.431805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:14:31.431901 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:14:31.432076 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:14:31.432076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:14:31.432078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:14:31.432140 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.432143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.433083 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.433084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.434228 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:14:31.434232 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.435020 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.435021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:14:31.435093 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.435096 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:14:31.435545 (+ 449us) client_negotiation.cc:240] Negotiation successful
0808 02:14:31.435589 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13715,"thread_start_us":82,"threads_started":1}
I20260808 02:14:31.436262 4461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d1687d95724856bfaa2cffd6c24ed9" candidate_uuid: "ea982a7ef94d4fd99c2058403afbbebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd5420345714178a9d713c6516e787d" is_pre_election: true
I20260808 02:14:31.436427 4461 raft_consensus.cc:2454] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea982a7ef94d4fd99c2058403afbbebf in term 0.
I20260808 02:14:31.436544 4459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d1687d95724856bfaa2cffd6c24ed9" candidate_uuid: "ea982a7ef94d4fd99c2058403afbbebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdd5420345714178a9d713c6516e787d"
I20260808 02:14:31.436614 4459 raft_consensus.cc:3037] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:14:31.437459 4459 raft_consensus.cc:2454] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea982a7ef94d4fd99c2058403afbbebf in term 1.
I20260808 02:14:31.445575 4729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.442025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48646 (local address 127.19.58.193:35373)
0808 02:14:31.442073 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 02:14:31.442075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:14:31.442086 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:14:31.442166 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:14:31.442167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:14:31.442184 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:14:31.442246 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:14:31.442424 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.443180 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.444557 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:14:31.444866 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:14:31.444921 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:14:31.445112 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:14:31.445233 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:14:31.445299 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:14:31.445458 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 02:14:31.445479 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:14:31.447525 4647 raft_consensus.cc:1261] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [term 1 FOLLOWER]: Refusing update from remote peer ea982a7ef94d4fd99c2058403afbbebf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:14:31.448098 4741 consensus_queue.cc:1035] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6284d745db0d4f7c943703ad6eb3413c" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:14:31.448248 4459 raft_consensus.cc:1261] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [term 1 FOLLOWER]: Refusing update from remote peer ea982a7ef94d4fd99c2058403afbbebf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:14:31.449915 4741 consensus_queue.cc:1035] T d4d1687d95724856bfaa2cffd6c24ed9 P ea982a7ef94d4fd99c2058403afbbebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdd5420345714178a9d713c6516e787d" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:14:31.454514 4029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:14:31.454604 4029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:14:31.455389 4029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50ccd28b844e449f840e40b0edb06e67: Sending DeleteTablet for 3 replicas of tablet d4d1687d95724856bfaa2cffd6c24ed9
I20260808 02:14:31.455804 4436 tablet_service.cc:1514] Processing DeleteTablet for tablet d4d1687d95724856bfaa2cffd6c24ed9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:14:31 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:57484
I20260808 02:14:31.456274 4226 tablet_service.cc:1514] Processing DeleteTablet for tablet d4d1687d95724856bfaa2cffd6c24ed9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:14:31 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:48638
I20260808 02:14:31.456745 4599 tablet_service.cc:1514] Processing DeleteTablet for tablet d4d1687d95724856bfaa2cffd6c24ed9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:14:31 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:47420
I20260808 02:14:31.457271 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 4140
I20260808 02:14:31.470875 4762 tablet_replica.cc:330] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: stopping tablet replica
I20260808 02:14:31.471067 4764 tablet_replica.cc:330] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: stopping tablet replica
I20260808 02:14:31.471271 4764 raft_consensus.cc:2229] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:14:31.471431 4762 raft_consensus.cc:2229] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:14:31.471562 4764 pending_rounds.cc:62] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Trying to abort 2 pending ops.
I20260808 02:14:31.471675 4762 pending_rounds.cc:62] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Trying to abort 2 pending ops.
I20260808 02:14:31.471778 4764 pending_rounds.cc:68] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316091991846371328 op_type: WRITE_OP write_request { tablet_id: "d4d1687d95724856bfaa2cffd6c24ed9" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\263\244\332\323\006\"8\n\ntest-admin\022*\n 51489002f649424da5dc2ba8e5ab9764\020\001\030\001 \001(\001" signature: "<redacted>""\010X\232\034\210\213\343\354j\025k\324\002\340\013\351\205\375\200\201\005\013[9B~\'q\317\222\360\230\243(P\270m\332s3\337\374;\367+hR\362 .bpZ\300\007\264\202\303@\0046\372m5" signing_key_seq_num: 0 } } request_id { client_id: "c97381e3d9c04299950e7d9c2ac145cf" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260808 02:14:31.472054 4762 pending_rounds.cc:68] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316091991846371328 op_type: WRITE_OP write_request { tablet_id: "d4d1687d95724856bfaa2cffd6c24ed9" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\263\244\332\323\006\"8\n\ntest-admin\022*\n 51489002f649424da5dc2ba8e5ab9764\020\001\030\001 \001(\001" signature: "<redacted>""\010X\232\034\210\213\343\354j\025k\324\002\340\013\351\205\375\200\201\005\013[9B~\'q\317\222\360\230\243(P\270m\332s3\337\374;\367+hR\362 .bpZ\300\007\264\202\303@\0046\372m5" signing_key_seq_num: 0 } } request_id { client_id: "c97381e3d9c04299950e7d9c2ac145cf" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260808 02:14:31.472393 4764 pending_rounds.cc:68] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316091991789199360 op_type: NO_OP noop_request { }
I20260808 02:14:31.472543 4762 pending_rounds.cc:68] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316091991789199360 op_type: NO_OP noop_request { }
I20260808 02:14:31.472637 4762 raft_consensus.cc:2866] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260808 02:14:31.472707 4762 raft_consensus.cc:2258] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:14:31.472774 4764 raft_consensus.cc:2866] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260808 02:14:31.472936 4764 raft_consensus.cc:2258] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:14:31.473495 4764 ts_tablet_manager.cc:1905] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:14:31.478755 4764 ts_tablet_manager.cc:1918] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260808 02:14:31.479565 4757 consensus_queue.cc:1159] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cdd5420345714178a9d713c6516e787d" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260808 02:14:31.480119 4762 ts_tablet_manager.cc:1905] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:14:31.478989 4764 log.cc:1196] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/d4d1687d95724856bfaa2cffd6c24ed9
I20260808 02:14:31.480911 4764 ts_tablet_manager.cc:1939] T d4d1687d95724856bfaa2cffd6c24ed9 P 6284d745db0d4f7c943703ad6eb3413c: Deleting consensus metadata
I20260808 02:14:31.481494 3999 catalog_manager.cc:4841] TS 6284d745db0d4f7c943703ad6eb3413c (127.19.58.195:46553): tablet d4d1687d95724856bfaa2cffd6c24ed9 (table test-table [id=51489002f649424da5dc2ba8e5ab9764]) successfully deleted
I20260808 02:14:31.481666 4762 ts_tablet_manager.cc:1918] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:14:31.481832 4762 log.cc:1196] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/d4d1687d95724856bfaa2cffd6c24ed9
I20260808 02:14:31.482133 4762 ts_tablet_manager.cc:1939] T d4d1687d95724856bfaa2cffd6c24ed9 P cdd5420345714178a9d713c6516e787d: Deleting consensus metadata
I20260808 02:14:31.482628 4003 catalog_manager.cc:4841] TS cdd5420345714178a9d713c6516e787d (127.19.58.194:35739): tablet d4d1687d95724856bfaa2cffd6c24ed9 (table test-table [id=51489002f649424da5dc2ba8e5ab9764]) successfully deleted
W20260808 02:14:31.483088 4001 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:35373 (error 108)
I20260808 02:14:31.483887 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 4346
I20260808 02:14:31.490978 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:14:31.490806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:35373 (local address 127.0.0.1:48660)
0808 02:14:31.490878 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 02:14:31.490936 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 02:14:31.491212 4001 catalog_manager.cc:4584] TS ea982a7ef94d4fd99c2058403afbbebf (127.19.58.193:35373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4d1687d95724856bfaa2cffd6c24ed9: Network error: Client connection negotiation failed: client connection to 127.19.58.193:35373: connect: Connection refused (error 111)
I20260808 02:14:31.494158 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 4531
I20260808 02:14:31.523927 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 3972
I20260808 02:14:31.552166 19691 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 02:14:31.897974 19691 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 02:14:31.898075 19691 mini_ranger.cc:66] Stopping Ranger...
I20260808 02:14:32.241609 19691 mini_ranger.cc:68] Stopped Ranger
2026-08-08 02:14:32.241 UTC [3369] LOG: received smart shutdown request
2026-08-08 02:14:32.245 UTC [3369] LOG: background worker "logical replication launcher" (PID 3381) exited with exit code 1
2026-08-08 02:14:32.245 UTC [3374] LOG: shutting down
2026-08-08 02:14:32.380 UTC [3369] LOG: database system is shut down
2026-08-08T02:14:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44395 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:14:32 dist-test-slave-1l3l krb5kdc[4775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:14:32 dist-test-slave-1l3l krb5kdc[4775](info): set up 2 sockets
Aug 08 02:14:32 dist-test-slave-1l3l krb5kdc[4775](info): commencing operation
krb5kdc: starting...
W20260808 02:14:33.587680 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:14:33 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:14:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:14:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.58.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.58.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/rangeradmin_127.19.58.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/rangeradmin_127.19.58.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.58.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.58.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/rangerlookup_127.19.58.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/rangerlookup_127.19.58.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.58.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.58.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/HTTP_127.19.58.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/HTTP_127.19.58.212@KRBTEST.COM.keytab.
I20260808 02:14:33.740238 19691 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDIPgti/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/postgres -l logfile start
2026-08-08 02:14:35.962 UTC [4815] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 02:14:35.962 UTC [4815] LOG: listening on IPv4 address "127.19.58.212", port 49085
2026-08-08 02:14:35.965 UTC [4815] LOG: listening on Unix socket "/tmp/.s.PGSQL.49085"
2026-08-08 02:14:35.993 UTC [4818] LOG: database system was shut down at 2026-08-08 02:14:34 UTC
2026-08-08 02:14:35.996 UTC [4815] LOG: database system is ready to accept connections
I20260808 02:14:37.025115 19691 mini_postgres.cc:96] Postgres bound to 49085
2026-08-08 02:14:37.033 UTC [4827] FATAL: database "slave" does not exist
127.19.58.212:49085 - accepting connections
I20260808 02:14:37.033870 19691 mini_ranger.cc:161] Starting Ranger...
I20260808 02:14:37.048982 19691 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 02:14:37.465353 19691 mini_ranger.cc:87] Created ranger Postgres database
I20260808 02:14:37.465440 19691 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-admin
2026-08-08 02:14:37,726 [I] DB FLAVOR :POSTGRES
2026-08-08 02:14:37,726 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:14:37,726 [I] Checking connection..
2026-08-08 02:14:37,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 02:14:37,977 [I] Checking connection passed.
2026-08-08 02:14:37,977 [I] --------- Verifying version history table ---------
2026-08-08 02:14:37,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:14:38,236 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:14:38,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:14:38,475 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 02:14:38,475 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 02:14:38,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 02:14:38.717 UTC [4908] WARNING: there is no transaction in progress
2026-08-08 02:14:38,725 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 02:14:38,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 02:14:38,977 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 02:14:38,978 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:14:38,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 02:14:39,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 02:14:39,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1l3l.c.gcp-upstream.internal','N') ;"
2026-08-08 02:14:39,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:14:39,920 [I] Table x_portal_user does not exist in database ranger
2026-08-08 02:14:39,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:14:40,148 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 02:14:40,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:14:40,383 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:14:40,638 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 02:14:40,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 02:14:41.520 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.534 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.548 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.562 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.576 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.595 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.647 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.653 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.666 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.675 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.684 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.696 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.704 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.709 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.715 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:41.720 UTC [5046] WARNING: there is no transaction in progress
2026-08-08 02:14:42,271 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 02:14:42,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 02:14:42,523 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 02:14:42,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 02:14:42,778 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 02:14:42,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 02:14:43,042 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 02:14:43,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 02:14:43,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 02:14:43,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1l3l.c.gcp-upstream.internal';"
2026-08-08 02:14:43,796 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 02:14:43,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 02:14:44,040 [I] DB_PATCHES have already been applied
I20260808 02:14:44.045521 19691 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDIPgti/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 02:14:44.045580 19691 mini_ranger.cc:193] Using host: 127.19.58.212
I20260808 02:14:44.049290 19691 mini_ranger.cc:227] Ranger admin URL: http://127.19.58.212:36083
Aug 08, 2026 2:14:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:14:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:14:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 2:14:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 2:14:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 2:14:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 2:14:44 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 02:14:45.235074 19691 mini_ranger.cc:160] Time spent starting Ranger: real 8.201s user 0.003s sys 0.007s
Aug 08, 2026 2:15:04 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 2:15:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 2:15:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 2:15:04 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 2:15:05 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 02:15:06.401530 19691 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.58.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.58.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/rangerkms_127.19.58.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.58.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/rangerkms_127.19.58.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.58.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.58.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.58.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/HTTP_127.19.58.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.58.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/HTTP_127.19.58.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 02:15:06 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155306, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 02:15:06.494722 19691 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 02:15:06.504716 19691 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 02:15:07.252036 19691 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 02:15:07.252118 19691 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-kms
I20260808 02:15:07.252135 19691 mini_ranger_kms.cc:227] Using postgres at 127.19.58.212:49085
2026-08-08 02:15:07,418 [I] DB FLAVOR :POSTGRES
2026-08-08 02:15:07,418 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 02:15:07,418 [I] Checking connection
2026-08-08 02:15:07,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 02:15:07,668 [I] connection success
2026-08-08 02:15:07,668 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 02:15:07,669 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 02:15:07,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 02:15:07,909 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 02:15:07,910 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 02:15:07,910 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 02:15:07,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.58.212:49085/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 02:15:08,162 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 02:15:08.629949 19691 mini_ranger_kms.cc:310] Created kms service
I20260808 02:15:08.821480 19691 mini_ranger_kms.cc:326] Created kudu user
I20260808 02:15:08.860396 19691 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 02:15:08 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155306, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:15:09.217923 19691 mini_ranger_kms.cc:384] Added ranger policy
I20260808 02:15:09.218058 19691 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDIPgti/build/release/bin/postgresql.jar:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDIPgti/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 02:15:09.218086 19691 mini_ranger_kms.cc:240] Using host: 127.19.58.212
I20260808 02:15:09.222568 19691 mini_ranger_kms.cc:276] Ranger KMS PID: 5290
I20260808 02:15:09.222630 19691 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.58.212:53231
Aug 08, 2026 2:15:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 2:15:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 2:15:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 2:15:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 2:15:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 2:15:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 02:15:11 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155311, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.58.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:11.669329 19691 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.175s user 0.000s sys 0.149s
I20260808 02:15:11.669438 19691 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 02:15:11.669488 19691 mini_ranger_kms.cc:401] 127.19.58.212:53231/kms/v1/keys
Aug 08 02:15:11 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155311, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.58.212@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
Aug 08 02:15:12 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155311, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.58.212@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
Aug 08 02:15:14 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:15:14.488459 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:45723
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:45723
--ranger_config_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:53231/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:14.568341 5360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:14.568534 5360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:14.568554 5360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:14.569651 5360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:14.569687 5360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:14.569700 5360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:14.569711 5360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:14.569727 5360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:14.571290 5360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.58.212:53231/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:45723
--ranger_config_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:45723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:14.571471 5360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:14.571636 5360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:14.576483 5370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:14.576524 5366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:14.576555 5360 server_base.cc:1034] running on GCE node
W20260808 02:15:14.576457 5367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:14.576921 5360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:14.577184 5360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:14.578337 5360 hybrid_clock.cc:648] HybridClock initialized: now 1786155314578312 us; error 42 us; skew 500 ppm
Aug 08 02:15:14 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155314, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:14.586619 5360 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:15:14.587059 5360 webserver.cc:460] Webserver started at http://127.19.58.254:35403/ using document root <none> and password file <none>
I20260808 02:15:14.587244 5360 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:14.587283 5360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:14.587404 5360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:15:14 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155314, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:15:14.634007 5360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "644a43207cd24817a86b5aac347ca0a2"
format_stamp: "Formatted at 2026-08-08 02:15:14 on dist-test-slave-1l3l"
server_key: "39cb61f4b50d63ac92d68966c6c5539c"
server_key_iv: "3c39ff4e009ca7faed7d65547a5bd57e"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:14.634454 5360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "644a43207cd24817a86b5aac347ca0a2"
format_stamp: "Formatted at 2026-08-08 02:15:14 on dist-test-slave-1l3l"
server_key: "39cb61f4b50d63ac92d68966c6c5539c"
server_key_iv: "3c39ff4e009ca7faed7d65547a5bd57e"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:14.635918 5360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:15:14.648775 5381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:14.648973 5360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:14.649076 5360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "644a43207cd24817a86b5aac347ca0a2"
format_stamp: "Formatted at 2026-08-08 02:15:14 on dist-test-slave-1l3l"
server_key: "39cb61f4b50d63ac92d68966c6c5539c"
server_key_iv: "3c39ff4e009ca7faed7d65547a5bd57e"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:14.649163 5360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:14.658766 5360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:14.660794 5360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:14.660940 5360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:14.665232 5360 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:45723
I20260808 02:15:14.665292 5449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:45723 every 8 connection(s)
I20260808 02:15:14.665635 5360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:14.666400 5451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:14.669616 5451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: Bootstrap starting.
I20260808 02:15:14.670320 5451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:14.670627 5451 log.cc:824] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:14.671360 5451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: No bootstrap required, opened a new log
I20260808 02:15:14.674003 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 5360
I20260808 02:15:14.674108 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:15:14.674274 19691 external_mini_cluster.cc:1428] Setting key 13e14bde9f274986b8fca34cecef79b6
I20260808 02:15:14.674144 5451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } }
I20260808 02:15:14.674314 5451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:14.674401 5451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 644a43207cd24817a86b5aac347ca0a2, State: Initialized, Role: FOLLOWER
I20260808 02:15:14.674582 5451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } }
I20260808 02:15:14.674749 5451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:14.674817 5451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:14.674876 5451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:14.675567 5451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } }
I20260808 02:15:14.675676 5451 leader_election.cc:302] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 644a43207cd24817a86b5aac347ca0a2; no voters:
I20260808 02:15:14.675849 5451 leader_election.cc:288] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:15:14.675909 5456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:14.676143 5456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 1 LEADER]: Becoming Leader. State: Replica: 644a43207cd24817a86b5aac347ca0a2, State: Running, Role: LEADER
I20260808 02:15:14.676225 5451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:14.676288 5456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } }
I20260808 02:15:14.676671 5456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "644a43207cd24817a86b5aac347ca0a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } } }
I20260808 02:15:14.676800 5456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:14.676971 5456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 644a43207cd24817a86b5aac347ca0a2. Latest consensus state: current_term: 1 leader_uuid: "644a43207cd24817a86b5aac347ca0a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } } }
I20260808 02:15:14.677071 5456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:14.677218 5464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:14.677724 5464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:14.678112 5451 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 08 02:15:14 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:14.686569 5464 catalog_manager.cc:1347] Generated new cluster ID: 51558dbfc6a946779b3e5e7e619d4962
I20260808 02:15:14.686653 5464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:14.688366 5455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:14.674725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.19.58.254:45723)
0808 02:15:14.675100 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:14.675107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:14.675133 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:15:14.675217 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:14.675223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:14.675263 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:14.675430 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:14.676197 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:14.677564 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:14.678587 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:14.679215 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:14.679325 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:14.682609 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:14.682660 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:14.682660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:14.682767 (+ 107us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:14.686270 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:14.686721 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:14.686723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:14.686724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:14.686868 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:14.687107 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:14.687108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:14.687108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:14.687408 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:14.687953 (+ 545us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:14.688019 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 02:15:14.688161 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":87,"threads_started":1}
I20260808 02:15:14.704298 5464 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:15:14.705137 5464 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:14.714651 5464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: Generated new TSK 0
I20260808 02:15:14.714959 5464 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:15:15.382539 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:53231/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:45723
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:15.483872 5510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:15.484149 5510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:15.484184 5510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:15.485476 5510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:15.485526 5510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:15.485562 5510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:15:15.487426 5510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.58.212:53231/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:45723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:15.487699 5510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:15.487970 5510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:15:15.493878 5510 server_base.cc:1034] running on GCE node
W20260808 02:15:15.493993 5520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:15.493856 5519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:15.496134 5522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:15.496479 5510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:15.496757 5510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:15.504137 5510 hybrid_clock.cc:648] HybridClock initialized: now 1786155315504103 us; error 49 us; skew 500 ppm
Aug 08 02:15:15 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155315, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:15.507767 5510 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:15:15.508369 5510 webserver.cc:460] Webserver started at http://127.19.58.193:41001/ using document root <none> and password file <none>
I20260808 02:15:15.508671 5510 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:15.508750 5510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:15.508910 5510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:15:15 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155315, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:15:15.523496 5510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "9ee7c7f51273444ca2c9bd1bdc44617e"
format_stamp: "Formatted at 2026-08-08 02:15:15 on dist-test-slave-1l3l"
server_key: "08ebad700ffc62e8964343078e2eba45"
server_key_iv: "a98714b506d6c41e14eb1088b174264c"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:15.523897 5510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "9ee7c7f51273444ca2c9bd1bdc44617e"
format_stamp: "Formatted at 2026-08-08 02:15:15 on dist-test-slave-1l3l"
server_key: "08ebad700ffc62e8964343078e2eba45"
server_key_iv: "a98714b506d6c41e14eb1088b174264c"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:15.525655 5510 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 02:15:15.541659 5536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:15.541927 5510 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 02:15:15.542052 5510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "9ee7c7f51273444ca2c9bd1bdc44617e"
format_stamp: "Formatted at 2026-08-08 02:15:15 on dist-test-slave-1l3l"
server_key: "08ebad700ffc62e8964343078e2eba45"
server_key_iv: "a98714b506d6c41e14eb1088b174264c"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:15.542164 5510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:15.558843 5510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:15.561265 5510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:15.561478 5510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:15.561779 5510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:15.562192 5510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:15.562283 5510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:15.562383 5510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:15.562469 5510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:15.569741 5510 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:38791
I20260808 02:15:15.570230 5510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:15.576018 5694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:38791 every 8 connection(s)
I20260808 02:15:15.578379 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 5510
I20260808 02:15:15.578475 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:15:15.578711 19691 external_mini_cluster.cc:1428] Setting key 22c1875a25d648c2bc69692da404906f
Aug 08 02:15:15 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155315, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:15.590342 5697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:15.572591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:45723 (local address 127.19.58.193:52031)
0808 02:15:15.579899 (+ 7308us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:15.579946 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:15.580650 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:15.581004 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:15.581009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:15.581254 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:15.581541 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:15.581547 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:15.582644 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:15.582645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:15.583915 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:15.583918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:15.584482 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:15.584483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:15.584620 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:15.585196 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:15.585206 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:15.586841 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:15.588786 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:15.588794 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:15.588796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:15.589071 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:15.589295 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:15.589295 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:15.589296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:15.589428 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:15.589706 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:15.589711 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:15.589864 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 02:15:15.589969 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 02:15:15.590116 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7263,"thread_start_us":89,"threads_started":1}
I20260808 02:15:15.590886 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:15.572888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:52031 (local address 127.19.58.254:45723)
0808 02:15:15.579825 (+ 6937us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:15.579827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:15.580537 (+ 710us) server_negotiation.cc:407] Connection header received
0808 02:15:15.580801 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:15.580802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:15.580816 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:15.580874 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:15.581647 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:15.582545 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:15.584009 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:15.584340 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:15.584406 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:15.586941 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:15.586943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:15.586943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:15.586950 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:15.588696 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:15.589150 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:15.589152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:15.589152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:15.589233 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:15.589494 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:15.589495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:15.589496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:15.589625 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:15.589682 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:15.590703 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0808 02:15:15.590791 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6902,"thread_start_us":80,"threads_started":1}
I20260808 02:15:15.591723 5695 heartbeater.cc:347] Connected to a master server at 127.19.58.254:45723
I20260808 02:15:15.591820 5695 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:15.592002 5695 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
I20260808 02:15:15.592604 5403 ts_manager.cc:194] Registered new tserver with Master: 9ee7c7f51273444ca2c9bd1bdc44617e (127.19.58.193:38791)
I20260808 02:15:15.593528 5403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:52031
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:15:15.619928 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:53231/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:45723
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:15:15 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155315, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 02:15:15.754309 5708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:15.754522 5708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:15.754555 5708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:15.755690 5708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:15.755757 5708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:15.755792 5708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:15:15.757297 5708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.58.212:53231/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:45723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:15.757536 5708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:15.757747 5708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:15.762562 5713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:15.762682 5708 server_base.cc:1034] running on GCE node
W20260808 02:15:15.762637 5714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:15.762774 5717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:15.763015 5708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:15.763237 5708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:15.764384 5708 hybrid_clock.cc:648] HybridClock initialized: now 1786155315764355 us; error 46 us; skew 500 ppm
Aug 08 02:15:15 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155315, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:15.766945 5708 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:15:15.767393 5708 webserver.cc:460] Webserver started at http://127.19.58.194:37459/ using document root <none> and password file <none>
I20260808 02:15:15.767622 5708 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:15.767673 5708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:15.767828 5708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:15:15 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155315, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:15:15.780874 5708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "d9dd5dec178e453da63e8b22edd3fba7"
format_stamp: "Formatted at 2026-08-08 02:15:15 on dist-test-slave-1l3l"
server_key: "5147453cdc821d645ffea738fc446d03"
server_key_iv: "72f153e25c0b7284113b096783bd1065"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:15.781426 5708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "d9dd5dec178e453da63e8b22edd3fba7"
format_stamp: "Formatted at 2026-08-08 02:15:15 on dist-test-slave-1l3l"
server_key: "5147453cdc821d645ffea738fc446d03"
server_key_iv: "72f153e25c0b7284113b096783bd1065"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:15.788122 5708 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260808 02:15:15.803251 5727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:15.803521 5708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:15.803618 5708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "d9dd5dec178e453da63e8b22edd3fba7"
format_stamp: "Formatted at 2026-08-08 02:15:15 on dist-test-slave-1l3l"
server_key: "5147453cdc821d645ffea738fc446d03"
server_key_iv: "72f153e25c0b7284113b096783bd1065"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:15.803738 5708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:15.824594 5708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:15.827605 5708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:15.827863 5708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:15.828131 5708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:15.828624 5708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:15.828722 5708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:15.828799 5708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:15.828855 5708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:15.835613 5708 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:35297
I20260808 02:15:15.836297 5708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:15.838651 5882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:35297 every 8 connection(s)
I20260808 02:15:15.840240 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 5708
I20260808 02:15:15.840349 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:15:15.840529 19691 external_mini_cluster.cc:1428] Setting key 7b6d6f16f6a8374e75d48d12d66e4729
Aug 08 02:15:15 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155315, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:15.849761 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:15.838240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:53073 (local address 127.19.58.254:45723)
0808 02:15:15.838293 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:15.838296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:15.839971 (+ 1675us) server_negotiation.cc:407] Connection header received
0808 02:15:15.840655 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:15.840657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:15.840675 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:15.840708 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:15.841543 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:15.842229 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:15.843623 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:15.843971 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:15.844013 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:15.846483 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:15.846485 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:15.846485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:15.846493 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:15.848203 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:15.848678 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:15.848679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:15.848679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:15.848738 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:15.849032 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:15.849033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:15.849033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:15.849166 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:15.849207 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:15.849578 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 02:15:15.849665 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 02:15:15.850247 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:15.837440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:45723 (local address 127.19.58.194:53073)
0808 02:15:15.839841 (+ 2401us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:15.839881 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:15.840524 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:15.840907 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:15.840912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:15.841153 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:15.841435 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:15.841442 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:15.842375 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:15.842376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:15.843524 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:15.843527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:15.844119 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:15.844120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:15.844254 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:15.844834 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:15.844844 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:15.846374 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:15.848300 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:15.848306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:15.848308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:15.848589 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:15.848809 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:15.848810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:15.848810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:15.848954 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:15.849254 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:15.849259 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:15.849418 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 02:15:15.849868 (+ 450us) client_negotiation.cc:240] Negotiation successful
0808 02:15:15.850022 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2336,"thread_start_us":577,"threads_started":1}
I20260808 02:15:15.850783 5883 heartbeater.cc:347] Connected to a master server at 127.19.58.254:45723
I20260808 02:15:15.850885 5883 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:15.851076 5883 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
I20260808 02:15:15.851559 5403 ts_manager.cc:194] Registered new tserver with Master: d9dd5dec178e453da63e8b22edd3fba7 (127.19.58.194:35297)
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
I20260808 02:15:15.852049 5403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:53073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:15:15.868561 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:53231/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:45723
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:15.986747 5890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:15.987075 5890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:15.987174 5890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:15.988912 5890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:15.989028 5890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:15.989111 5890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:15:15.991207 5890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.58.212:53231/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:45723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:15.991792 5890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:15.992127 5890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:15.998787 5899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:15.999200 5898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:15.999435 5902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:16.000578 5890 server_base.cc:1034] running on GCE node
I20260808 02:15:16.000890 5890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:16.001147 5890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:16.002302 5890 hybrid_clock.cc:648] HybridClock initialized: now 1786155316002283 us; error 33 us; skew 500 ppm
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:16.005656 5890 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:15:16.006166 5890 webserver.cc:460] Webserver started at http://127.19.58.195:38259/ using document root <none> and password file <none>
I20260808 02:15:16.006454 5890 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:16.006511 5890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:16.006640 5890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:15:16.021417 5890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5"
format_stamp: "Formatted at 2026-08-08 02:15:16 on dist-test-slave-1l3l"
server_key: "d0d71329a6e1c93f79efd7c15737814b"
server_key_iv: "9b2262f13f0d0d533059fbbfb15416dc"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:16.021811 5890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5"
format_stamp: "Formatted at 2026-08-08 02:15:16 on dist-test-slave-1l3l"
server_key: "d0d71329a6e1c93f79efd7c15737814b"
server_key_iv: "9b2262f13f0d0d533059fbbfb15416dc"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:16.023257 5890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:15:16.035408 5910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.035588 5890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:15:16.035679 5890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5"
format_stamp: "Formatted at 2026-08-08 02:15:16 on dist-test-slave-1l3l"
server_key: "d0d71329a6e1c93f79efd7c15737814b"
server_key_iv: "9b2262f13f0d0d533059fbbfb15416dc"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:16.035794 5890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:16.069348 5890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:16.071744 5890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:16.071908 5890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:16.072175 5890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:16.072527 5890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:16.072572 5890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.072608 5890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:16.072625 5890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.079130 5890 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:39185
I20260808 02:15:16.079557 5890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:15:16.082055 6052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:39185 every 8 connection(s)
I20260808 02:15:16.085955 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 5890
I20260808 02:15:16.086047 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:15:16.086233 19691 external_mini_cluster.cc:1428] Setting key fafd39038ccbe31553c5fdeb7d1dab61
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:16.094355 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:16.083390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:33029 (local address 127.19.58.254:45723)
0808 02:15:16.083435 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:16.083437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:16.083632 (+ 195us) server_negotiation.cc:407] Connection header received
0808 02:15:16.084347 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:16.084350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:16.084363 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:16.084393 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:16.085218 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.085988 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.088118 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.088413 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.088452 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:16.091121 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:16.091123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:16.091123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:16.091130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:16.092815 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:16.093328 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:16.093330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:16.093330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:16.093389 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:16.093645 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:16.093645 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:16.093646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:16.093770 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:16.093807 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:16.094165 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 02:15:16.094254 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:15:16.094889 6057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:16.083051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:45723 (local address 127.19.58.195:33029)
0808 02:15:16.083508 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:16.083553 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:16.084220 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:16.084576 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:16.084581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:16.084823 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:16.085110 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:16.085115 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.086104 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.086105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:16.088019 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:16.088022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.088552 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.088553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:16.088696 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:16.089304 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:16.089323 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:16.091007 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:16.092917 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:16.092924 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:16.092926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:16.093205 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:16.093456 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:16.093457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:16.093457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:16.093579 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:16.093856 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:16.093862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:16.094019 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 02:15:16.094491 (+ 472us) client_negotiation.cc:240] Negotiation successful
0808 02:15:16.094650 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":145,"threads_started":1}
I20260808 02:15:16.095290 6053 heartbeater.cc:347] Connected to a master server at 127.19.58.254:45723
I20260808 02:15:16.095404 6053 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:16.095592 6053 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
I20260808 02:15:16.095964 5403 ts_manager.cc:194] Registered new tserver with Master: ee6ad3ceb1274e539e4ec0fb256db5b5 (127.19.58.195:39185)
I20260808 02:15:16.096442 5403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:33029
I20260808 02:15:16.097697 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:15:16.098512 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 5510
I20260808 02:15:16.108801 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 5708
I20260808 02:15:16.121709 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 5890
I20260808 02:15:16.129558 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 5360
I20260808 02:15:16.141885 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:45723
--webserver_interface=127.19.58.254
--webserver_port=35403
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:45723
--ranger_config_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:53231/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.58.212:53231/kms with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:16.222880 6058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:16.223083 6058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:16.223112 6058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:16.224268 6058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:16.224310 6058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:16.224337 6058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:16.224354 6058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:16.224370 6058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:16.225946 6058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.58.212:53231/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:45723
--ranger_config_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:45723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=35403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:16.226181 6058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:16.226387 6058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:16.231001 6067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:16.231011 6065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:16.231170 6064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:16.231153 6058 server_base.cc:1034] running on GCE node
I20260808 02:15:16.231504 6058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:16.231828 6058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:16.232997 6058 hybrid_clock.cc:648] HybridClock initialized: now 1786155316232980 us; error 33 us; skew 500 ppm
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:16.235677 6058 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:15:16.236126 6058 webserver.cc:460] Webserver started at http://127.19.58.254:35403/ using document root <none> and password file <none>
I20260808 02:15:16.236326 6058 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:16.236383 6058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:15:16.307505 6058 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:16.308477 6079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.308633 6058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:16.308717 6058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "644a43207cd24817a86b5aac347ca0a2"
format_stamp: "Formatted at 2026-08-08 02:15:14 on dist-test-slave-1l3l"
server_key: "39cb61f4b50d63ac92d68966c6c5539c"
server_key_iv: "3c39ff4e009ca7faed7d65547a5bd57e"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:16.309178 6058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:16.338380 6058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:16.339946 6058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:16.340062 6058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:16.344054 6058 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:45723
I20260808 02:15:16.344116 6147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:45723 every 8 connection(s)
I20260808 02:15:16.344460 6058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:16.345676 6148 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:15:16.346305 6148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: Bootstrap starting.
I20260808 02:15:16.348109 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 6058
I20260808 02:15:16.348376 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:53231/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:38791
--local_ip_for_outbound_sockets=127.19.58.193
--tserver_master_addrs=127.19.58.254:45723
--webserver_port=41001
--webserver_interface=127.19.58.193
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.58.212:53231/kms with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:16.354650 6148 log.cc:824] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:16.356390 6148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:16.356644 6148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: Bootstrap complete.
I20260808 02:15:16.358892 6148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } }
I20260808 02:15:16.359119 6148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 644a43207cd24817a86b5aac347ca0a2, State: Initialized, Role: FOLLOWER
I20260808 02:15:16.359256 6148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } }
I20260808 02:15:16.359328 6148 raft_consensus.cc:388] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:16.359371 6148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:16.359414 6148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:15:16.360446 6148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } }
I20260808 02:15:16.360533 6148 leader_election.cc:302] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 644a43207cd24817a86b5aac347ca0a2; no voters:
I20260808 02:15:16.360689 6148 leader_election.cc:288] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:15:16.360743 6154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:15:16.360863 6154 raft_consensus.cc:686] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [term 2 LEADER]: Becoming Leader. State: Replica: 644a43207cd24817a86b5aac347ca0a2, State: Running, Role: LEADER
I20260808 02:15:16.360935 6148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:16.360945 6154 consensus_queue.cc:237] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } }
I20260808 02:15:16.361307 6158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 644a43207cd24817a86b5aac347ca0a2. Latest consensus state: current_term: 2 leader_uuid: "644a43207cd24817a86b5aac347ca0a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } } }
I20260808 02:15:16.361398 6158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:16.361322 6156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "644a43207cd24817a86b5aac347ca0a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "644a43207cd24817a86b5aac347ca0a2" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45723 } } }
I20260808 02:15:16.361555 6156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:16.362047 6162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:16.362442 6148 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 02:15:16.362538 6162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:16.362659 6162 catalog_manager.cc:1259] Loaded cluster ID: 51558dbfc6a946779b3e5e7e619d4962
I20260808 02:15:16.362699 6162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:16.366590 6162 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:16.367146 6162 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: Loaded TSK: 0
I20260808 02:15:16.367869 6162 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:15:16.457726 6150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:16.457938 6150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:16.457973 6150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:16.459206 6150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:16.459252 6150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:16.459285 6150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:15:16.460860 6150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.58.212:53231/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:38791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=41001
--tserver_master_addrs=127.19.58.254:45723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:16.461093 6150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:16.461328 6150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:16.466634 6199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:16.466610 6196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:16.466827 6197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:16.467263 6150 server_base.cc:1034] running on GCE node
I20260808 02:15:16.467588 6150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:16.467841 6150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:16.469019 6150 hybrid_clock.cc:648] HybridClock initialized: now 1786155316469001 us; error 26 us; skew 500 ppm
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:16.471628 6150 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:15:16.472157 6150 webserver.cc:460] Webserver started at http://127.19.58.193:41001/ using document root <none> and password file <none>
I20260808 02:15:16.472379 6150 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:16.472442 6150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:15:16.498452 6150 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.001s
I20260808 02:15:16.499631 6210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.499856 6150 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:15:16.499945 6150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "9ee7c7f51273444ca2c9bd1bdc44617e"
format_stamp: "Formatted at 2026-08-08 02:15:15 on dist-test-slave-1l3l"
server_key: "08ebad700ffc62e8964343078e2eba45"
server_key_iv: "a98714b506d6c41e14eb1088b174264c"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:16.500412 6150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:16.527885 6150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:16.530130 6150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:16.530329 6150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:16.530578 6150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:16.530938 6150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:16.531008 6150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.531060 6150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:16.531100 6150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.537446 6150 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:38791
I20260808 02:15:16.537822 6367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:38791 every 8 connection(s)
I20260808 02:15:16.537976 6150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:16.547019 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 6150
I20260808 02:15:16.547302 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:53231/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:35297
--local_ip_for_outbound_sockets=127.19.58.194
--tserver_master_addrs=127.19.58.254:45723
--webserver_port=37459
--webserver_interface=127.19.58.194
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.58.212:53231/kms with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:16.550909 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:16.538843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:41635 (local address 127.19.58.254:45723)
0808 02:15:16.539053 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:16.539062 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:16.539184 (+ 122us) server_negotiation.cc:407] Connection header received
0808 02:15:16.540001 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:16.540009 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:16.540066 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:16.540214 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:16.540878 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.542229 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.543636 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.544259 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.544362 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:16.547137 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:16.547141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:16.547142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:16.547152 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:16.549006 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:16.549487 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:16.549491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:16.549491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:16.549599 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:16.549856 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:16.549857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:16.549857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:16.550175 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:16.550261 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:16.550598 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 02:15:16.550722 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":79,"threads_started":1}
I20260808 02:15:16.551117 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:16.538759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:45723 (local address 127.19.58.193:41635)
0808 02:15:16.539016 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:16.539062 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:16.539759 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:16.540268 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:16.540272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:16.540496 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:16.540754 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:16.540760 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.542349 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.542350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:16.543547 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:16.543549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.544376 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.544378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:16.544519 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:16.545079 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:16.545088 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:16.547009 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:16.549094 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:16.549101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:16.549103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:16.549391 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:16.549669 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:16.549670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:16.549670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:16.549785 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:16.550246 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:16.550252 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:16.550411 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 02:15:16.550530 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 02:15:16.550675 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":84,"threads_started":1}
I20260808 02:15:16.552134 6368 heartbeater.cc:347] Connected to a master server at 127.19.58.254:45723
I20260808 02:15:16.552235 6368 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:16.552435 6368 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
W20260808 02:15:16.553393 6368 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:45723 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 02:15:16.553475 6368 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:16.553546 6368 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
I20260808 02:15:16.553902 6368 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:16.553987 6368 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
W20260808 02:15:16.554317 6368 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:15:16.652215 6374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:16.652415 6374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:16.652448 6374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:16.653688 6374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:16.653741 6374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:16.653776 6374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:15:16.655586 6374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.58.212:53231/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:35297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=37459
--tserver_master_addrs=127.19.58.254:45723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:16.655848 6374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:16.656085 6374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:16.662282 6380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:16.662866 6385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:16.662901 6382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:16.663430 6374 server_base.cc:1034] running on GCE node
I20260808 02:15:16.663676 6374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:16.663971 6374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:16.665154 6374 hybrid_clock.cc:648] HybridClock initialized: now 1786155316665131 us; error 37 us; skew 500 ppm
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:16.667886 6374 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:15:16.668435 6374 webserver.cc:460] Webserver started at http://127.19.58.194:37459/ using document root <none> and password file <none>
I20260808 02:15:16.668684 6374 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:16.668764 6374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:15:16.684741 6374 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:16.685568 6395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.685731 6374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 02:15:16.685819 6374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "d9dd5dec178e453da63e8b22edd3fba7"
format_stamp: "Formatted at 2026-08-08 02:15:15 on dist-test-slave-1l3l"
server_key: "5147453cdc821d645ffea738fc446d03"
server_key_iv: "72f153e25c0b7284113b096783bd1065"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:16.686158 6374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:16.738415 6374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:16.741010 6374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:16.741254 6374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:16.741631 6374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:16.742090 6374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:16.742148 6374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.742182 6374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:16.742265 6374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.750267 6374 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:35297
I20260808 02:15:16.750303 6550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:35297 every 8 connection(s)
I20260808 02:15:16.750823 6374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:16.755055 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 6374
I20260808 02:15:16.755347 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.58.212:53231/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:39185
--local_ip_for_outbound_sockets=127.19.58.195
--tserver_master_addrs=127.19.58.254:45723
--webserver_port=38259
--webserver_interface=127.19.58.195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:41869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.58.212:53231/kms with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:16.763651 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:16.751580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:54975 (local address 127.19.58.254:45723)
0808 02:15:16.751635 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:16.751638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:16.753666 (+ 2028us) server_negotiation.cc:407] Connection header received
0808 02:15:16.754305 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:16.754307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:16.754319 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:16.754364 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:16.755082 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.755907 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.757381 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.757677 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.757724 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:16.760238 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:16.760239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:16.760239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:16.760248 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:16.761953 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:16.762502 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:16.762504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:16.762504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:16.762578 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:16.762880 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:16.762881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:16.762881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:16.763026 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:16.763072 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:16.763468 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 02:15:16.763558 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:15:16.764428 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:16.751590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:45723 (local address 127.19.58.194:54975)
0808 02:15:16.753517 (+ 1927us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:16.753560 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:16.754199 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:16.754453 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:16.754458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:16.754681 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:16.754956 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:16.754963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.756022 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.756023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:16.757278 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:16.757281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.757836 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.757837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:16.757974 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:16.758501 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:16.758511 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:16.760126 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:16.762059 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:16.762066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:16.762068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:16.762366 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:16.762658 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:16.762659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:16.762659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:16.762795 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:16.763122 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:16.763129 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:16.763285 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 02:15:16.764019 (+ 734us) client_negotiation.cc:240] Negotiation successful
0808 02:15:16.764184 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1887,"thread_start_us":1734,"threads_started":1}
I20260808 02:15:16.765087 6551 heartbeater.cc:347] Connected to a master server at 127.19.58.254:45723
I20260808 02:15:16.765187 6551 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:16.765381 6551 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
W20260808 02:15:16.766002 6551 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:45723 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 02:15:16.766078 6551 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:16.766137 6551 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
I20260808 02:15:16.766562 6551 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:16.766639 6551 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
W20260808 02:15:16.766978 6551 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:15:16.850987 6554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:16.851181 6554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:16.851210 6554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:16.852348 6554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:16.852391 6554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:16.852424 6554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:15:16.853847 6554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:41869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.58.212:53231/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:39185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=38259
--tserver_master_addrs=127.19.58.254:45723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:16.854061 6554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:16.854281 6554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:16.859279 6565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:16.859278 6562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:16.859829 6554 server_base.cc:1034] running on GCE node
W20260808 02:15:16.859869 6561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:16.860092 6554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:16.860316 6554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:16.861469 6554 hybrid_clock.cc:648] HybridClock initialized: now 1786155316861456 us; error 27 us; skew 500 ppm
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:16.864214 6554 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:15:16.864645 6554 webserver.cc:460] Webserver started at http://127.19.58.195:38259/ using document root <none> and password file <none>
I20260808 02:15:16.864846 6554 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:16.864908 6554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:15:16.893817 6554 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 02:15:16.898736 6579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.898981 6554 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260808 02:15:16.899092 6554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5"
format_stamp: "Formatted at 2026-08-08 02:15:16 on dist-test-slave-1l3l"
server_key: "d0d71329a6e1c93f79efd7c15737814b"
server_key_iv: "9b2262f13f0d0d533059fbbfb15416dc"
server_key_version: "kuduclusterkey@0"
I20260808 02:15:16.899497 6554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:16.925028 6554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:16.927284 6554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:16.927515 6554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:16.927855 6554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:16.928229 6554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:16.928329 6554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.928409 6554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:16.928470 6554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:16.935653 6554 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:39185
I20260808 02:15:16.935758 6740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:39185 every 8 connection(s)
I20260808 02:15:16.936086 6554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
Aug 08 02:15:16 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155316, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:16.944072 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 6554
I20260808 02:15:16.946188 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:16.936840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:48179 (local address 127.19.58.254:45723)
0808 02:15:16.936885 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:16.936887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:16.937126 (+ 239us) server_negotiation.cc:407] Connection header received
0808 02:15:16.937613 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:16.937615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:16.937628 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:16.937703 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:16.938402 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.939149 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.940592 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.940894 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.940977 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:16.943065 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:16.943067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:16.943067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:16.943075 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:16.944658 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:16.945097 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:16.945098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:16.945098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:16.945163 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:16.945444 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:16.945444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:16.945445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:16.945578 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:16.945621 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:16.946005 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 02:15:16.946093 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:15:16.946537 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:16.936848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:45723 (local address 127.19.58.195:48179)
0808 02:15:16.937011 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:16.937046 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:16.937506 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:16.937783 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:16.937787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:16.938001 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:16.938275 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:16.938283 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.939251 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.939252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:16.940474 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:16.940479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.940999 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.941000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:16.941099 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:16.941564 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:16.941571 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:16.942948 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:16.944739 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:16.944744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:16.944746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:16.945015 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:16.945234 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:16.945234 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:16.945235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:16.945368 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:16.945671 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:16.945678 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:16.945834 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 02:15:16.945943 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 02:15:16.946103 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260808 02:15:16.946991 6742 heartbeater.cc:347] Connected to a master server at 127.19.58.254:45723
I20260808 02:15:16.947134 6742 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:16.947355 6742 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
W20260808 02:15:16.948350 6742 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:45723 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 02:15:16.948421 6742 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:16.948511 6742 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
I20260808 02:15:16.948863 6742 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:16.948944 6742 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
W20260808 02:15:16.949467 6742 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:15:16.951502 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:16.944778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.19.58.254:45723)
0808 02:15:16.944913 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:16.944915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:16.944923 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:15:16.944943 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:16.944943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:16.944955 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:16.945769 (+ 814us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:16.945802 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.946518 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.947382 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:16.947591 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:16.947920 (+ 329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:16.948992 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:16.948993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:16.948993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:16.948999 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:16.950387 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:16.950764 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:16.950765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:16.950766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:16.950825 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:16.951009 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:16.951009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:16.951010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:16.951116 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:16.951281 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:16.951316 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 02:15:16.951403 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
Aug 08 02:15:17 dist-test-slave-1l3l krb5kdc[4775](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786155317, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:17.554507 6368 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:17.554663 6368 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
I20260808 02:15:17.555081 6102 ts_manager.cc:194] Registered new tserver with Master: 9ee7c7f51273444ca2c9bd1bdc44617e (127.19.58.193:38791)
I20260808 02:15:17.556118 6102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:41635
I20260808 02:15:17.767124 6551 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:17.767275 6551 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
I20260808 02:15:17.767621 6101 ts_manager.cc:194] Registered new tserver with Master: d9dd5dec178e453da63e8b22edd3fba7 (127.19.58.194:35297)
I20260808 02:15:17.768193 6101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:54975
Aug 08 02:15:17 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155317, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
I20260808 02:15:17.949653 6742 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:17.949813 6742 heartbeater.cc:511] Master 127.19.58.254:45723 requested a full tablet report, sending...
I20260808 02:15:17.950133 6102 ts_manager.cc:194] Registered new tserver with Master: ee6ad3ceb1274e539e4ec0fb256db5b5 (127.19.58.195:39185)
I20260808 02:15:17.950680 6102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:48179
I20260808 02:15:17.951612 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:15:17.960163 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:17.952644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40644 (local address 127.19.58.254:45723)
0808 02:15:17.952770 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:17.952772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:17.952782 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:15:17.952829 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:17.952830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:17.952843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:17.952936 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:17.953606 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.954361 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.955384 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.955692 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.955779 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:17.957234 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:17.957235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:17.957235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:17.957243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:17.958958 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:17.959343 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:17.959344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:17.959344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:17.959414 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:17.959600 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:17.959600 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:17.959601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:17.959764 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:17.959867 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:17.959975 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 02:15:17.960063 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260808 02:15:17.962414 6102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:15:17.964697 6102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:15:17.974939 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:17.969436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40760 (local address 127.19.58.194:35297)
0808 02:15:17.969765 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:17.969770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:17.969991 (+ 221us) server_negotiation.cc:407] Connection header received
0808 02:15:17.970163 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:17.970165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:17.970183 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:17.970234 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:17.970503 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.971519 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.973329 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.974428 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.974474 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:17.974538 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:17.974814 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 02:15:17.974856 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":78,"threads_started":1}
I20260808 02:15:17.975431 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:17.967619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35297 (local address 127.0.0.1:40760)
0808 02:15:17.969924 (+ 2305us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:17.969934 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:17.970018 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:17.970357 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:17.970360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:17.970362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:17.970423 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:17.970425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.971619 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.971619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:17.973209 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:17.973212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.974607 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.974608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:17.974693 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:17.974707 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 02:15:17.975263 (+ 556us) client_negotiation.cc:240] Negotiation successful
0808 02:15:17.975350 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2288,"thread_start_us":91,"threads_started":1}
I20260808 02:15:17.975735 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:17.969796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:38791 (local address 127.0.0.1:36842)
0808 02:15:17.971813 (+ 2017us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:17.971823 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:17.971890 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:17.972171 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:17.972171 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:17.972173 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:17.972215 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:17.972217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.973443 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.973444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:17.974472 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:17.974476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.975495 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.975497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:17.975555 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:17.975559 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:15:17.975605 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:15:17.975640 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1998,"thread_start_us":73,"threads_started":1}
I20260808 02:15:17.976079 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:17.969681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36842 (local address 127.19.58.193:38791)
0808 02:15:17.970116 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:17.970121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:17.971952 (+ 1831us) server_negotiation.cc:407] Connection header received
0808 02:15:17.971989 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:17.971992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:17.972008 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:17.972074 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:17.972286 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.973345 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.974568 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.975326 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.975405 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:17.975459 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:17.975967 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 02:15:17.976007 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":52,"threads_started":1}
I20260808 02:15:17.976379 6465 tablet_service.cc:1467] Processing CreateTablet for tablet 5a7ffeafd07f4796be98f77f5a0df604 (DEFAULT_TABLE table=test-table [id=a2d3b540e34049babd64599fdee4e248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:17.976804 6465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a7ffeafd07f4796be98f77f5a0df604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:17.976969 6278 tablet_service.cc:1467] Processing CreateTablet for tablet 5a7ffeafd07f4796be98f77f5a0df604 (DEFAULT_TABLE table=test-table [id=a2d3b540e34049babd64599fdee4e248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:17.977288 6278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a7ffeafd07f4796be98f77f5a0df604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:17.980990 6792 tablet_bootstrap.cc:492] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e: Bootstrap starting.
I20260808 02:15:17.981103 6795 tablet_bootstrap.cc:492] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: Bootstrap starting.
I20260808 02:15:17.981683 6792 tablet_bootstrap.cc:654] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:17.981858 6795 tablet_bootstrap.cc:654] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:17.982084 6792 log.cc:824] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:17.982210 6795 log.cc:824] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:17.983098 6795 tablet_bootstrap.cc:492] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: No bootstrap required, opened a new log
I20260808 02:15:17.983393 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:17.967590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:39185 (local address 127.0.0.1:54352)
0808 02:15:17.968030 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:17.968174 (+ 144us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:17.968230 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:17.979998 (+ 11768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:17.979999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:17.980001 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:17.980042 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:17.980043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.981290 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.981291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:17.982298 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:17.982302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.983154 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.983155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:17.983219 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:17.983223 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:15:17.983285 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 02:15:17.983324 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":84,"threads_started":1}
I20260808 02:15:17.983618 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:17.967867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.19.58.195:39185)
0808 02:15:17.979816 (+ 11949us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:17.979822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:17.979836 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:15:17.979870 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:17.979872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:17.979890 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:17.980122 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:17.980210 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.981190 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.982403 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:17.983029 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:17.983143 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:17.983208 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:17.983500 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 02:15:17.983544 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11895,"thread_start_us":60,"threads_started":1}
I20260808 02:15:17.983233 6795 ts_tablet_manager.cc:1397] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:15:17.984395 6648 tablet_service.cc:1467] Processing CreateTablet for tablet 5a7ffeafd07f4796be98f77f5a0df604 (DEFAULT_TABLE table=test-table [id=a2d3b540e34049babd64599fdee4e248]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:17.984738 6648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a7ffeafd07f4796be98f77f5a0df604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:17.986558 6795 raft_consensus.cc:348] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 } } peers { permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 } } peers { permanent_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38791 } }
I20260808 02:15:17.986794 6795 raft_consensus.cc:374] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:17.986873 6795 raft_consensus.cc:729] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9dd5dec178e453da63e8b22edd3fba7, State: Initialized, Role: FOLLOWER
I20260808 02:15:17.987040 6795 consensus_queue.cc:260] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 } } peers { permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 } } peers { permanent_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38791 } }
I20260808 02:15:17.987367 6795 ts_tablet_manager.cc:1428] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:17.987516 6551 heartbeater.cc:503] Master 127.19.58.254:45723 was elected leader, sending a full tablet report...
I20260808 02:15:17.988278 6800 tablet_bootstrap.cc:492] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: Bootstrap starting.
I20260808 02:15:17.988916 6800 tablet_bootstrap.cc:654] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:17.989250 6800 log.cc:824] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:17.990018 6800 tablet_bootstrap.cc:492] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: No bootstrap required, opened a new log
I20260808 02:15:17.990075 6800 ts_tablet_manager.cc:1397] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:15:17.992317 6792 tablet_bootstrap.cc:492] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e: No bootstrap required, opened a new log
I20260808 02:15:17.992376 6792 ts_tablet_manager.cc:1397] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260808 02:15:17.992630 6800 raft_consensus.cc:348] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 } } peers { permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 } } peers { permanent_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38791 } }
I20260808 02:15:17.992769 6800 raft_consensus.cc:374] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:17.992796 6800 raft_consensus.cc:729] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee6ad3ceb1274e539e4ec0fb256db5b5, State: Initialized, Role: FOLLOWER
I20260808 02:15:17.993260 6800 consensus_queue.cc:260] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 } } peers { permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 } } peers { permanent_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38791 } }
I20260808 02:15:17.993623 6800 ts_tablet_manager.cc:1428] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 02:15:17.993765 6742 heartbeater.cc:503] Master 127.19.58.254:45723 was elected leader, sending a full tablet report...
I20260808 02:15:17.994819 6792 raft_consensus.cc:348] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 } } peers { permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 } } peers { permanent_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38791 } }
I20260808 02:15:17.994988 6792 raft_consensus.cc:374] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:17.995040 6792 raft_consensus.cc:729] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ee7c7f51273444ca2c9bd1bdc44617e, State: Initialized, Role: FOLLOWER
I20260808 02:15:17.995154 6792 consensus_queue.cc:260] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 } } peers { permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 } } peers { permanent_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38791 } }
I20260808 02:15:17.995435 6792 ts_tablet_manager.cc:1428] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:17.995474 6368 heartbeater.cc:503] Master 127.19.58.254:45723 was elected leader, sending a full tablet report...
W20260808 02:15:18.002027 6552 tablet.cc:2369] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:15:18.039498 6369 tablet.cc:2369] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 02:15:18 dist-test-slave-1l3l krb5kdc[4775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155317, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for HTTP/127.19.58.212@KRBTEST.COM
W20260808 02:15:18.187288 6743 tablet.cc:2369] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:15:18.265033 6803 raft_consensus.cc:482] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:18.265225 6803 raft_consensus.cc:504] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 } } peers { permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 } } peers { permanent_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38791 } }
I20260808 02:15:18.265681 6803 leader_election.cc:288] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee6ad3ceb1274e539e4ec0fb256db5b5 (127.19.58.195:39185), d9dd5dec178e453da63e8b22edd3fba7 (127.19.58.194:35297)
I20260808 02:15:18.272377 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:18.266542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:41551 (local address 127.19.58.195:39185)
0808 02:15:18.266592 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:18.266594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:18.266769 (+ 175us) server_negotiation.cc:407] Connection header received
0808 02:15:18.266837 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:18.266839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:18.266849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:18.266903 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:18.267120 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:18.267899 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:18.270496 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:18.271062 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:18.271111 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:18.271120 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:18.272244 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0808 02:15:18.272291 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 02:15:18.272606 6809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:18.265984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:39185 (local address 127.19.58.193:41551)
0808 02:15:18.266703 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:18.266715 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:18.266784 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:18.266958 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:18.266959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:18.266960 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:18.267042 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:18.267044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:18.268000 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:18.268002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:18.269255 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:18.269258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:18.271194 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:18.271195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:18.271265 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:18.271269 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:15:18.272468 (+ 1199us) client_negotiation.cc:240] Negotiation successful
0808 02:15:18.272517 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":100,"threads_started":1}
I20260808 02:15:18.272964 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:18.266626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:46783 (local address 127.19.58.194:35297)
0808 02:15:18.266686 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:18.266688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:18.266930 (+ 242us) server_negotiation.cc:407] Connection header received
0808 02:15:18.267022 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:18.267023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:18.267032 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:18.267057 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:18.267236 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:18.267938 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:18.271361 (+ 3423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:18.271961 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:18.272008 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:18.272015 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:18.272852 (+ 837us) server_negotiation.cc:299] Negotiation successful
0808 02:15:18.272902 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:15:18.272969 6672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a7ffeafd07f4796be98f77f5a0df604" candidate_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" is_pre_election: true
I20260808 02:15:18.273118 6672 raft_consensus.cc:2454] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ee7c7f51273444ca2c9bd1bdc44617e in term 0.
I20260808 02:15:18.273365 6211 leader_election.cc:302] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ee7c7f51273444ca2c9bd1bdc44617e, ee6ad3ceb1274e539e4ec0fb256db5b5; no voters:
I20260808 02:15:18.273504 6803 raft_consensus.cc:2781] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:18.273550 6803 raft_consensus.cc:482] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:18.273576 6803 raft_consensus.cc:3037] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:18.274461 6803 raft_consensus.cc:504] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 } } peers { permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 } } peers { permanent_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38791 } }
I20260808 02:15:18.274600 6803 leader_election.cc:288] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [CANDIDATE]: Term 1 election: Requested vote from peers ee6ad3ceb1274e539e4ec0fb256db5b5 (127.19.58.195:39185), d9dd5dec178e453da63e8b22edd3fba7 (127.19.58.194:35297)
I20260808 02:15:18.274782 6672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a7ffeafd07f4796be98f77f5a0df604" candidate_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5"
I20260808 02:15:18.274861 6672 raft_consensus.cc:3037] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:18.275693 6672 raft_consensus.cc:2454] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ee7c7f51273444ca2c9bd1bdc44617e in term 1.
I20260808 02:15:18.275875 6211 leader_election.cc:302] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ee7c7f51273444ca2c9bd1bdc44617e, ee6ad3ceb1274e539e4ec0fb256db5b5; no voters:
I20260808 02:15:18.275954 6803 raft_consensus.cc:2781] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:18.276104 6803 raft_consensus.cc:686] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 1 LEADER]: Becoming Leader. State: Replica: 9ee7c7f51273444ca2c9bd1bdc44617e, State: Running, Role: LEADER
I20260808 02:15:18.276194 6803 consensus_queue.cc:237] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 } } peers { permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 } } peers { permanent_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38791 } }
I20260808 02:15:18.276980 6102 catalog_manager.cc:5495] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ee7c7f51273444ca2c9bd1bdc44617e (127.19.58.193). New cstate: current_term: 1 leader_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38791 } health_report { overall_health: HEALTHY } } }
I20260808 02:15:18.273181 6807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:18.266215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35297 (local address 127.19.58.193:46783)
0808 02:15:18.266880 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:18.266888 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:18.266954 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:18.267116 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:18.267116 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:18.267118 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:18.267170 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:18.267171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:18.268045 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:18.268046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:18.270386 (+ 2340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:18.270389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:18.272071 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:18.272071 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:18.272162 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:18.272165 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:15:18.273073 (+ 908us) client_negotiation.cc:240] Negotiation successful
0808 02:15:18.273111 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":1199,"thread_start_us":67,"threads_started":1}
I20260808 02:15:18.278301 6489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a7ffeafd07f4796be98f77f5a0df604" candidate_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9dd5dec178e453da63e8b22edd3fba7" is_pre_election: true
I20260808 02:15:18.278456 6489 raft_consensus.cc:2454] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ee7c7f51273444ca2c9bd1bdc44617e in term 0.
I20260808 02:15:18.278577 6488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a7ffeafd07f4796be98f77f5a0df604" candidate_uuid: "9ee7c7f51273444ca2c9bd1bdc44617e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9dd5dec178e453da63e8b22edd3fba7"
I20260808 02:15:18.278636 6488 raft_consensus.cc:3037] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:18.279412 6488 raft_consensus.cc:2454] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ee7c7f51273444ca2c9bd1bdc44617e in term 1.
I20260808 02:15:18.328361 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:18.324983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45368 (local address 127.19.58.193:38791)
0808 02:15:18.325023 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:18.325025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:18.325035 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:15:18.325142 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:18.325143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:18.325165 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:18.325237 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:15:18.325461 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:18.326212 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:18.327267 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:18.327573 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:18.327633 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:18.327873 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:15:18.328006 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:15:18.328077 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:18.328246 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 02:15:18.328272 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:15:18.330421 6488 raft_consensus.cc:1261] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7 [term 1 FOLLOWER]: Refusing update from remote peer 9ee7c7f51273444ca2c9bd1bdc44617e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:18.330499 6672 raft_consensus.cc:1261] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5 [term 1 FOLLOWER]: Refusing update from remote peer 9ee7c7f51273444ca2c9bd1bdc44617e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:18.330834 6810 consensus_queue.cc:1035] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee6ad3ceb1274e539e4ec0fb256db5b5" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:18.330961 6803 consensus_queue.cc:1035] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9dd5dec178e453da63e8b22edd3fba7" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:18.334040 6825 mvcc.cc:205] Tried to move back new op lower bound from 7316092183878377472 to 7316092183659712512. Current Snapshot: MvccSnapshot[applied={T|T < 7316092183878377472 or (T in {7316092183878377472})}]
I20260808 02:15:18.336040 6101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:18.336129 6101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:18.336621 6826 mvcc.cc:205] Tried to move back new op lower bound from 7316092183878377472 to 7316092183659712512. Current Snapshot: MvccSnapshot[applied={T|T < 7316092183878377472 or (T in {7316092183878377472})}]
I20260808 02:15:18.336824 6827 mvcc.cc:205] Tried to move back new op lower bound from 7316092183878377472 to 7316092183659712512. Current Snapshot: MvccSnapshot[applied={T|T < 7316092183878377472}]
I20260808 02:15:18.336896 6101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 644a43207cd24817a86b5aac347ca0a2: Sending DeleteTablet for 3 replicas of tablet 5a7ffeafd07f4796be98f77f5a0df604
I20260808 02:15:18.337230 6648 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a7ffeafd07f4796be98f77f5a0df604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:18 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:54352
I20260808 02:15:18.337419 6465 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a7ffeafd07f4796be98f77f5a0df604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:18 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:40760
I20260808 02:15:18.337476 6278 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a7ffeafd07f4796be98f77f5a0df604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:18 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:36842
I20260808 02:15:18.337698 6841 tablet_replica.cc:330] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e: stopping tablet replica
I20260808 02:15:18.337800 6841 raft_consensus.cc:2229] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:15:18.337881 6841 raft_consensus.cc:2258] T 5a7ffeafd07f4796be98f77f5a0df604 P 9ee7c7f51273444ca2c9bd1bdc44617e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:18.338176 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 6150
I20260808 02:15:18.345757 6839 tablet_replica.cc:330] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: stopping tablet replica
I20260808 02:15:18.345885 6839 raft_consensus.cc:2229] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:18.345981 6839 raft_consensus.cc:2258] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:18.346503 6839 ts_tablet_manager.cc:1905] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:18.348125 6840 tablet_replica.cc:330] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: stopping tablet replica
I20260808 02:15:18.348212 6839 ts_tablet_manager.cc:1918] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:18.348248 6840 raft_consensus.cc:2229] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:18.348305 6839 log.cc:1196] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/5a7ffeafd07f4796be98f77f5a0df604
I20260808 02:15:18.348377 6840 raft_consensus.cc:2258] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:18.348578 6839 ts_tablet_manager.cc:1939] T 5a7ffeafd07f4796be98f77f5a0df604 P ee6ad3ceb1274e539e4ec0fb256db5b5: Deleting consensus metadata
I20260808 02:15:18.348860 6840 ts_tablet_manager.cc:1905] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:18.348973 6080 catalog_manager.cc:4841] TS ee6ad3ceb1274e539e4ec0fb256db5b5 (127.19.58.195:39185): tablet 5a7ffeafd07f4796be98f77f5a0df604 (table test-table [id=a2d3b540e34049babd64599fdee4e248]) successfully deleted
W20260808 02:15:18.350257 6084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:38791 (error 108)
I20260808 02:15:18.350494 6840 ts_tablet_manager.cc:1918] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:18.350569 6840 log.cc:1196] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/5a7ffeafd07f4796be98f77f5a0df604
I20260808 02:15:18.350715 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:18.350608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:38791 (local address 127.0.0.1:45378)
0808 02:15:18.350655 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:18.350688 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:38791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 02:15:18.350834 6840 ts_tablet_manager.cc:1939] T 5a7ffeafd07f4796be98f77f5a0df604 P d9dd5dec178e453da63e8b22edd3fba7: Deleting consensus metadata
W20260808 02:15:18.350838 6084 catalog_manager.cc:4584] TS 9ee7c7f51273444ca2c9bd1bdc44617e (127.19.58.193:38791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a7ffeafd07f4796be98f77f5a0df604: Network error: Client connection negotiation failed: client connection to 127.19.58.193:38791: connect: Connection refused (error 111)
I20260808 02:15:18.351228 6082 catalog_manager.cc:4841] TS d9dd5dec178e453da63e8b22edd3fba7 (127.19.58.194:35297): tablet 5a7ffeafd07f4796be98f77f5a0df604 (table test-table [id=a2d3b540e34049babd64599fdee4e248]) successfully deleted
I20260808 02:15:18.351238 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 6374
I20260808 02:15:18.360176 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 6554
I20260808 02:15:18.371376 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 6058
I20260808 02:15:18.383575 19691 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 02:15:18.727936 19691 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 02:15:18.728031 19691 mini_ranger.cc:66] Stopping Ranger...
I20260808 02:15:19.071574 19691 mini_ranger.cc:68] Stopped Ranger
2026-08-08 02:15:19.071 UTC [4815] LOG: received smart shutdown request
2026-08-08 02:15:19.075 UTC [4815] LOG: background worker "logical replication launcher" (PID 4824) exited with exit code 1
2026-08-08 02:15:19.076 UTC [4819] LOG: shutting down
2026-08-08 02:15:19.209 UTC [4815] LOG: database system is shut down
2026-08-08T02:15:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46825 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:15:19 dist-test-slave-1l3l krb5kdc[6856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:15:19 dist-test-slave-1l3l krb5kdc[6856](info): set up 2 sockets
Aug 08 02:15:19 dist-test-slave-1l3l krb5kdc[6856](info): commencing operation
krb5kdc: starting...
W20260808 02:15:20.412930 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:15:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:15:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:15:20.526361 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:46773
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:46773,127.19.58.253:40967,127.19.58.252:34089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:20.607110 6876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:20.607481 6876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:20.607554 6876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:20.608932 6876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:20.608981 6876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:20.609002 6876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:20.609018 6876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:20.609035 6876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:20.610576 6876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:46773,127.19.58.253:40967,127.19.58.252:34089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:46773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:20.610819 6876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:20.611019 6876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:20.615690 6885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:20.615744 6883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:20.615758 6881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:20.615849 6876 server_base.cc:1034] running on GCE node
I20260808 02:15:20.616174 6876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:20.616425 6876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:20.617573 6876 hybrid_clock.cc:648] HybridClock initialized: now 1786155320617560 us; error 30 us; skew 500 ppm
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:20.622361 6876 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:15:20.622784 6876 webserver.cc:460] Webserver started at http://127.19.58.254:36005/ using document root <none> and password file <none>
I20260808 02:15:20.622959 6876 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:20.622999 6876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:20.623090 6876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:20.624109 6876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "11f60db7d86c4845bb9239567b329537"
format_stamp: "Formatted at 2026-08-08 02:15:20 on dist-test-slave-1l3l"
server_key: "ae1cb7441a1f07ad4eeccf364f51f70c"
server_key_iv: "3f45032b534aa5c4217b9eca15cd441a"
server_key_version: "encryptionkey@0"
I20260808 02:15:20.624408 6876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "11f60db7d86c4845bb9239567b329537"
format_stamp: "Formatted at 2026-08-08 02:15:20 on dist-test-slave-1l3l"
server_key: "ae1cb7441a1f07ad4eeccf364f51f70c"
server_key_iv: "3f45032b534aa5c4217b9eca15cd441a"
server_key_version: "encryptionkey@0"
I20260808 02:15:20.625600 6876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:15:20.626214 6894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:20.626362 6876 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:15:20.626456 6876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "11f60db7d86c4845bb9239567b329537"
format_stamp: "Formatted at 2026-08-08 02:15:20 on dist-test-slave-1l3l"
server_key: "ae1cb7441a1f07ad4eeccf364f51f70c"
server_key_iv: "3f45032b534aa5c4217b9eca15cd441a"
server_key_version: "encryptionkey@0"
I20260808 02:15:20.626531 6876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:20.637758 6876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:20.639586 6876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:20.639734 6876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:20.643630 6876 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:46773
I20260808 02:15:20.643666 6959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:46773 every 8 connection(s)
I20260808 02:15:20.644037 6876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:20.644824 6960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:20.646399 6960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:15:20.650686 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 6876
I20260808 02:15:20.650777 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:15:20.650964 19691 external_mini_cluster.cc:1428] Setting key 84369d6e30352d8764c6e51c657bdd26
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:20.655320 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.647170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.19.58.254:46773)
0808 02:15:20.647280 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.647284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.647312 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:15:20.647782 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.647784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.647800 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.647891 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.648413 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.649207 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.650016 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.650317 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.650399 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.652394 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.652404 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.652404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.652411 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.653834 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.654223 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.654225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.654225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.654296 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.654507 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.654507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.654508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.654697 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.654752 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.654996 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.655085 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":39,"threads_started":1}
I20260808 02:15:20.655326 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.647018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:46773 (local address 127.0.0.1:39064)
0808 02:15:20.647185 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.647220 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:20.647646 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:20.647889 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:20.647891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:20.648146 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:20.648295 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.648298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.649283 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.649283 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.649951 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.649952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.650374 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.650374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.650430 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.650953 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:20.650961 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:20.652316 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:20.653912 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.653919 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.653921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.654155 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.654348 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.654349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.654349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.654450 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.654759 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:20.654767 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:20.654849 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 02:15:20.654962 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 02:15:20.655179 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":63,"thread_start_us":66,"threads_started":1}
I20260808 02:15:20.656083 6960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:15:20.656430 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.656311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:40967 (local address 127.0.0.1:57446)
0808 02:15:20.656357 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.656406 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 02:15:20.656561 6896 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.58.253:40967: connect: Connection refused (error 111)
I20260808 02:15:20.656961 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.656862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:40967 (local address 127.0.0.1:57458)
0808 02:15:20.656910 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.656939 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 02:15:20.657059 6960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.253:40967: Network error: Client connection negotiation failed: client connection to 127.19.58.253:40967: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.19.58.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab.
Entry for principal kudu/127.19.58.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab.
I20260808 02:15:20.674777 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.253:40967
--webserver_interface=127.19.58.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab
--principal=kudu/127.19.58.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:46773,127.19.58.253:40967,127.19.58.252:34089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:20.706261 6960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } attempt: 1
I20260808 02:15:20.706758 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.706595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:40967 (local address 127.0.0.1:57474)
0808 02:15:20.706664 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.706716 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 02:15:20.707113 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.707017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:40967 (local address 127.0.0.1:57490)
0808 02:15:20.707062 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.707092 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 02:15:20.707206 6960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.253:40967: Network error: Client connection negotiation failed: client connection to 127.19.58.253:40967: connect: Connection refused (error 111)
W20260808 02:15:20.754055 6969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:20.754245 6969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:20.754266 6969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:20.755366 6969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:20.755410 6969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:20.755424 6969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:20.755437 6969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:20.755461 6969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:20.756930 6969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:46773,127.19.58.253:40967,127.19.58.252:34089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab
--principal=kudu/127.19.58.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.253:40967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.58.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:20.757143 6969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:20.757339 6969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:20.761858 6975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:20.761858 6983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:20.761875 6977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:20.761911 6969 server_base.cc:1034] running on GCE node
I20260808 02:15:20.762198 6969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:20.762466 6969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:20.763620 6969 hybrid_clock.cc:648] HybridClock initialized: now 1786155320763609 us; error 27 us; skew 500 ppm
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:20.766032 6969 init.cc:377] Logged in from keytab as kudu/127.19.58.253@KRBTEST.COM (short username kudu)
I20260808 02:15:20.766485 6969 webserver.cc:460] Webserver started at http://127.19.58.253:35531/ using document root <none> and password file <none>
I20260808 02:15:20.766682 6969 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:20.766729 6969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:20.766853 6969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:20.767889 6969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/data/instance:
uuid: "abbc7e2d67e64237b10722e5c306bedb"
format_stamp: "Formatted at 2026-08-08 02:15:20 on dist-test-slave-1l3l"
server_key: "c29b74266501788436df6233d450605c"
server_key_iv: "d8cffcdb84a98f3d74f262580c75900e"
server_key_version: "encryptionkey@0"
I20260808 02:15:20.768206 6969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/wal/instance:
uuid: "abbc7e2d67e64237b10722e5c306bedb"
format_stamp: "Formatted at 2026-08-08 02:15:20 on dist-test-slave-1l3l"
server_key: "c29b74266501788436df6233d450605c"
server_key_iv: "d8cffcdb84a98f3d74f262580c75900e"
server_key_version: "encryptionkey@0"
I20260808 02:15:20.769436 6969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:20.770067 6990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:20.770215 6969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:15:20.770303 6969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/wal
uuid: "abbc7e2d67e64237b10722e5c306bedb"
format_stamp: "Formatted at 2026-08-08 02:15:20 on dist-test-slave-1l3l"
server_key: "c29b74266501788436df6233d450605c"
server_key_iv: "d8cffcdb84a98f3d74f262580c75900e"
server_key_version: "encryptionkey@0"
I20260808 02:15:20.770358 6969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:20.775350 6960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } attempt: 2
I20260808 02:15:20.775724 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.775588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:40967 (local address 127.0.0.1:57494)
0808 02:15:20.775640 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.775675 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 02:15:20.776011 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.775940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:40967 (local address 127.0.0.1:57510)
0808 02:15:20.775972 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.775995 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 02:15:20.776085 6960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.253:40967: Network error: Client connection negotiation failed: client connection to 127.19.58.253:40967: connect: Connection refused (error 111)
I20260808 02:15:20.788446 6969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:20.790133 6969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:20.790244 6969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:20.794589 6969 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.253:40967
I20260808 02:15:20.794628 7062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.253:40967 every 8 connection(s)
I20260808 02:15:20.795073 6969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
I20260808 02:15:20.796017 7064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:20.797480 7064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:15:20.799304 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 6969
I20260808 02:15:20.799383 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-1/wal/instance
I20260808 02:15:20.799536 19691 external_mini_cluster.cc:1428] Setting key e8b15e0c4f2b52ae1cf54819fe7a4a76
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:20.806288 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.798164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39070 (local address 127.19.58.254:46773)
0808 02:15:20.798243 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.798246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.798411 (+ 165us) server_negotiation.cc:407] Connection header received
0808 02:15:20.798882 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.798883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.798909 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.798964 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.799592 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.800135 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.801113 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.801350 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.801412 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.803558 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.803559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.803559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.803568 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.805036 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.805403 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.805404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.805404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.805468 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.805670 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.805671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.805671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.805809 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.805872 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.806104 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.806196 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 02:15:20.806329 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.798104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:46773 (local address 127.0.0.1:39070)
0808 02:15:20.798293 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.798334 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:20.798789 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:20.799039 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:20.799043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:20.799279 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:20.799506 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.799511 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.800222 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.800223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.801008 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.801012 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.801437 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.801438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.801582 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.802072 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:20.802082 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:20.803449 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:20.805112 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.805119 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.805121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.805334 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.805527 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.805527 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.805528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.805618 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.805874 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:20.805877 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:20.805989 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 02:15:20.806064 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 02:15:20.806175 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260808 02:15:20.806855 7064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.58.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.252@KRBTEST.COM" created.
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
I20260808 02:15:20.813654 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.807084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:40967 (local address 127.0.0.1:57512)
0808 02:15:20.807124 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.807133 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:20.807165 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:20.807409 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:20.807410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:20.807560 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:20.807611 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.807613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.808475 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.808476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.809050 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.809052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.809486 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.809487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.809551 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.809958 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:20.809962 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:20.811110 (+ 1148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:20.812751 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.812752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.812753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.812902 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.813072 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.813073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.813073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.813126 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.813363 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:20.813364 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:20.813408 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 02:15:20.813469 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 02:15:20.813562 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 02:15:20.813745 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.807165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57512 (local address 127.19.58.253:40967)
0808 02:15:20.807299 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.807303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.807317 (+ 14us) server_negotiation.cc:407] Connection header received
0808 02:15:20.807330 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.807332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.807341 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.807402 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.807677 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.808391 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.809114 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.809419 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.809478 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.811182 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.811189 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.811190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.811196 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.812679 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.812967 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.812970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.812970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.813022 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.813173 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.813173 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.813174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.813305 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.813374 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.813515 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.813633 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":113,"thread_start_us":90,"threads_started":1}
I20260808 02:15:20.814298 7064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:15:20.814599 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.814498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50042)
0808 02:15:20.814535 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.814577 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 02:15:20.814704 6992 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
I20260808 02:15:20.815071 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.814990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50052)
0808 02:15:20.815025 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.815048 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 02:15:20.815164 7064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:34089: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab.
Entry for principal kudu/127.19.58.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab.
I20260808 02:15:20.824433 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.252:34089
--webserver_interface=127.19.58.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab
--principal=kudu/127.19.58.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:46773,127.19.58.253:40967,127.19.58.252:34089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:20.864377 7064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } attempt: 1
I20260808 02:15:20.864924 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.864753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50054)
0808 02:15:20.864819 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.864880 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 02:15:20.865262 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.865177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50068)
0808 02:15:20.865216 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.865243 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 02:15:20.865343 7064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:34089: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
I20260808 02:15:20.867234 6960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } attempt: 3
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
I20260808 02:15:20.874341 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.867544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:40967 (local address 127.0.0.1:57520)
0808 02:15:20.867608 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.867620 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:20.867677 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:20.867844 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:20.867845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:20.868003 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:20.868070 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.868072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.868996 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.868998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.869540 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.869542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.870056 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.870057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.870108 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.870534 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:20.870540 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:20.871812 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:20.873341 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.873342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.873343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.873559 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.873749 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.873750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.873750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.873816 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.874032 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:20.874033 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:20.874082 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 02:15:20.874144 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 02:15:20.874245 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 02:15:20.874339 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.867595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.19.58.253:40967)
0808 02:15:20.867640 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.867643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.867659 (+ 16us) server_negotiation.cc:407] Connection header received
0808 02:15:20.867754 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.867755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.867766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.867819 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.868133 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.868901 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.869636 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.869989 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.870050 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.871890 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.871892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.871892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.871898 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.873270 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.873624 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.873624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.873625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.873673 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.873869 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.873870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.873871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.873981 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.874036 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.874172 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.874246 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:15:20.874709 6960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:15:20.875007 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.874918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50070)
0808 02:15:20.874956 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.874982 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 02:15:20.875330 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.875247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50084)
0808 02:15:20.875285 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.875314 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 02:15:20.875413 6960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:34089: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
W20260808 02:15:20.905225 7073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:20.905435 7073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:20.905457 7073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:20.906507 7073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:20.906543 7073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:20.906563 7073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:20.906589 7073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:20.906605 7073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:20.906589 6960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } attempt: 1
I20260808 02:15:20.906987 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.906867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50092)
0808 02:15:20.906923 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.906956 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 02:15:20.907357 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.907279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50100)
0808 02:15:20.907318 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.907342 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 02:15:20.907476 6960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:34089: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
I20260808 02:15:20.908277 7073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:46773,127.19.58.253:40967,127.19.58.252:34089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab
--principal=kudu/127.19.58.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.252:34089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.58.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:20.908478 7073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:20.908649 7073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:20.913205 7085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:20.913235 7083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:20.913256 7073 server_base.cc:1034] running on GCE node
W20260808 02:15:20.913239 7082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:20.913640 7073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:20.913877 7073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:20.915020 7073 hybrid_clock.cc:648] HybridClock initialized: now 1786155320915010 us; error 27 us; skew 500 ppm
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:20.917379 7073 init.cc:377] Logged in from keytab as kudu/127.19.58.252@KRBTEST.COM (short username kudu)
I20260808 02:15:20.917797 7073 webserver.cc:460] Webserver started at http://127.19.58.252:37087/ using document root <none> and password file <none>
I20260808 02:15:20.917997 7073 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:20.918046 7073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:20.918169 7073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:20.919252 7073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/data/instance:
uuid: "dc8c1c452e884e54b2094635d04ce7a9"
format_stamp: "Formatted at 2026-08-08 02:15:20 on dist-test-slave-1l3l"
server_key: "c64afe833e1b0252c6d2e0e47f04faf5"
server_key_iv: "edcd181eb809ffd4b0649637dfe51a0d"
server_key_version: "encryptionkey@0"
I20260808 02:15:20.919584 7073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/wal/instance:
uuid: "dc8c1c452e884e54b2094635d04ce7a9"
format_stamp: "Formatted at 2026-08-08 02:15:20 on dist-test-slave-1l3l"
server_key: "c64afe833e1b0252c6d2e0e47f04faf5"
server_key_iv: "edcd181eb809ffd4b0649637dfe51a0d"
server_key_version: "encryptionkey@0"
I20260808 02:15:20.920817 7073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:15:20.921469 7095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:20.921612 7073 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:15:20.921693 7073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/wal
uuid: "dc8c1c452e884e54b2094635d04ce7a9"
format_stamp: "Formatted at 2026-08-08 02:15:20 on dist-test-slave-1l3l"
server_key: "c64afe833e1b0252c6d2e0e47f04faf5"
server_key_iv: "edcd181eb809ffd4b0649637dfe51a0d"
server_key_version: "encryptionkey@0"
I20260808 02:15:20.921753 7073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:20.933493 7064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } attempt: 2
I20260808 02:15:20.933921 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.933790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50106)
0808 02:15:20.933855 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.933892 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 02:15:20.934252 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.934167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50110)
0808 02:15:20.934207 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.934230 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 02:15:20.934331 7064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:34089: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34089: connect: Connection refused (error 111)
I20260808 02:15:20.934937 7073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:20.936782 7073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:20.936921 7073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:20.940587 7073 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.252:34089
I20260808 02:15:20.940618 7171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.252:34089 every 8 connection(s)
I20260808 02:15:20.940927 7073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
I20260808 02:15:20.941695 7172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:20.943142 7172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:20.948926 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 7073
I20260808 02:15:20.949020 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786155170542577-19691-0/minicluster-data/master-2/wal/instance
I20260808 02:15:20.949203 19691 external_mini_cluster.cc:1428] Setting key ec60d4a914312878ecf8cace552ed0df
I20260808 02:15:20.951439 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.943778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.19.58.254:46773)
0808 02:15:20.943835 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.943837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.944033 (+ 196us) server_negotiation.cc:407] Connection header received
0808 02:15:20.944447 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.944448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.944459 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.944510 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.945039 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.945522 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.946436 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.946618 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.946669 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.948611 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.948613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.948613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.948621 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.950164 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.950580 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.950581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.950582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.950642 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.950843 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.950843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.950844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.950976 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.951039 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.951239 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.951321 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 02:15:20.951532 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.943742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:46773 (local address 127.0.0.1:39078)
0808 02:15:20.943920 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.943959 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:20.944372 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:20.944559 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:20.944562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:20.944789 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:20.944978 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.944982 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.945594 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.945596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.946346 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.946348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.946679 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.946679 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.946774 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.947206 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:20.947213 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:20.948523 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:20.950236 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.950244 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.950246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.950520 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.950700 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.950701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.950701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.950792 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.951032 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:20.951035 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:20.951136 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 02:15:20.951206 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 02:15:20.951369 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":50,"thread_start_us":89,"threads_started":1}
I20260808 02:15:20.952049 7172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
I20260808 02:15:20.958155 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.949578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50116 (local address 127.19.58.252:34089)
0808 02:15:20.949750 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.949754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.949801 (+ 47us) server_negotiation.cc:407] Connection header received
0808 02:15:20.949824 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.949826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.949838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.949936 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.950536 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.951487 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.952479 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.952865 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.952920 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.955328 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.955335 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.955335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.955342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.956994 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.957316 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.957319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.957320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.957376 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.957551 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.957552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.957552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.957714 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.957834 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.957911 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.957975 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260808 02:15:20.958271 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.952246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:40967 (local address 127.0.0.1:57524)
0808 02:15:20.952284 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.952293 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:20.952326 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:20.952474 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:20.952475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:20.952608 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:20.952655 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.952656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.953363 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.953364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.954146 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.954149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.954493 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.954494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.954566 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.954851 (+ 285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:20.954855 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:20.955943 (+ 1088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:20.957372 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.957373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.957373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.957535 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.957691 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.957692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.957692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.957734 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.957984 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:20.957985 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:20.958021 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 02:15:20.958075 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 02:15:20.958157 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 02:15:20.958268 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.952309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.19.58.253:40967)
0808 02:15:20.952350 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.952352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.952359 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:15:20.952383 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.952384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.952393 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.952468 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.952723 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.953274 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.954228 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.954423 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.954475 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.956025 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.956026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.956026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.956034 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.957302 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.957583 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.957584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.957585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.957636 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.957809 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.957810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.957810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.957926 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.957985 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.958108 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.958176 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 02:15:20.958693 7172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:20.965197 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.958925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50124)
0808 02:15:20.958961 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.958969 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:20.958999 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:20.959254 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:20.959255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:20.959378 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:20.959409 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.959411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.960185 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.960185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.960812 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.960814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.961132 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.961133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.961184 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.961518 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:20.961521 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:20.962652 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:20.964322 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.964323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.964323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.964483 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.964664 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.964665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.964665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.964708 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.964908 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:20.964909 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:20.964948 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 02:15:20.965021 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 02:15:20.965103 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 02:15:20.965265 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.958963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50124 (local address 127.19.58.252:34089)
0808 02:15:20.958993 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.958995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.959002 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:15:20.959043 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.959044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.959053 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.959232 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.959479 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.960119 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.960892 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.961074 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.961121 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.962721 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.962722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.962723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.962729 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.964253 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.964550 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.964551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.964552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.964605 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.964744 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.964744 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.964745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.964850 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.964915 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.965039 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.965154 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":14}
I20260808 02:15:20.965845 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.959072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.19.58.254:46773)
0808 02:15:20.959112 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.959114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.959134 (+ 20us) server_negotiation.cc:407] Connection header received
0808 02:15:20.959152 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.959152 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.959159 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.959189 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.959686 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.960331 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.961131 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.961332 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.961378 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.963494 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.963495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.963495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.963501 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.964812 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.965135 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.965135 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.965136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.965187 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.965368 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.965369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.965369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.965488 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.965654 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.965691 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.965766 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 02:15:20.966624 7172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9: Bootstrap starting.
I20260808 02:15:20.967394 7172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:20.967625 7172 log.cc:824] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:20.968425 7172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9: No bootstrap required, opened a new log
I20260808 02:15:20.970156 7172 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } }
I20260808 02:15:20.970278 7172 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:20.970311 7172 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc8c1c452e884e54b2094635d04ce7a9, State: Initialized, Role: FOLLOWER
I20260808 02:15:20.970444 7172 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } }
I20260808 02:15:20.970700 7180 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } } }
I20260808 02:15:20.970736 7172 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:20.970788 7180 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
W20260808 02:15:20.972631 7193 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:15:20.972770 7193 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:15:20.975063 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.968161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.19.58.253:40967)
0808 02:15:20.968195 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.968197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.968205 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:15:20.968248 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.968248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.968255 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.968293 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.968795 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.969206 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.970063 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.970224 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.970262 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.972621 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.972622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.972622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.972629 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.974106 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.974387 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.974388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.974389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.974437 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.974593 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.974594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.974594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.974693 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.974866 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.974914 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.974980 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 02:15:20.976290 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:15:20.973642 7073 master_runner.cc:186] Error getting master registration for 127.19.58.254:46773: OK, instance_id { permanent_uuid: "11f60db7d86c4845bb9239567b329537" instance_seqno: 1786155320642562 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:15:20.976392 7073 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:15:20.982595 6960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } attempt: 2
I20260808 02:15:20.984850 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.977518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.19.58.254:46773)
0808 02:15:20.977553 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.977555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.977666 (+ 111us) server_negotiation.cc:407] Connection header received
0808 02:15:20.977757 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.977758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.977767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.977792 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.978497 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.979171 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.980179 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.980457 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.980493 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.981916 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.981918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.981918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.981924 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.983625 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.984036 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.984037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.984037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.984085 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.984268 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.984269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.984269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.984385 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.984470 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.984687 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.984763 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 02:15:20.985052 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.977712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57534 (local address 127.19.58.253:40967)
0808 02:15:20.977743 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.977745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.978001 (+ 256us) server_negotiation.cc:407] Connection header received
0808 02:15:20.978046 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.978047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.978056 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.978105 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.978703 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.979348 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.980264 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.980525 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.980559 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.981939 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.981940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.981941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.981947 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.983440 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.983826 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.983827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.983827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.983877 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.984085 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.984085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.984085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.984199 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.984696 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.984887 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.984975 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:15:20.986603 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.977850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.19.58.252:34089)
0808 02:15:20.977879 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.977881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.978122 (+ 241us) server_negotiation.cc:407] Connection header received
0808 02:15:20.978205 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.978205 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.978215 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.978244 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.978956 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.979634 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.980649 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.981364 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.981409 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.984084 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.984085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.984085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.984091 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.985614 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.985975 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.985976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.985976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.986026 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.986195 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.986195 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.986195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.986306 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.986381 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.986492 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.986549 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Aug 08 02:15:20 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
I20260808 02:15:20.992678 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.983433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50148 (local address 127.19.58.252:34089)
0808 02:15:20.986662 (+ 3229us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:20.986664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:20.986670 (+ 6us) server_negotiation.cc:407] Connection header received
0808 02:15:20.986686 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:20.986687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:20.986696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:20.986747 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:20.987093 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.987605 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.988574 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.988740 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.988789 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.990404 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:20.990405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:20.990405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:20.990411 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:20.991759 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.992091 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.992092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.992092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.992137 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:20.992298 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:20.992298 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:20.992298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:20.992378 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:20.992434 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:20.992571 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 02:15:20.992623 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3216,"thread_start_us":83,"threads_started":1}
I20260808 02:15:20.992729 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:20.983224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50148)
0808 02:15:20.983262 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:20.983271 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:20.983301 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:20.986787 (+ 3486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:20.986789 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:20.986939 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:20.986975 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.986976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.987696 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.987697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.988503 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:20.988505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:20.988801 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:20.988802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:20.988855 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:20.989177 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:20.989181 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:20.990325 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:20.991816 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.991817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.991817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.992029 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.992192 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:20.992193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:20.992193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:20.992250 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:20.992442 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:20.992443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:20.992484 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:15:20.992556 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:15:20.992637 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 02:15:20.994355 6960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537: Bootstrap starting.
I20260808 02:15:20.995164 6960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:20.995476 6960 log.cc:824] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:20.996308 6960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537: No bootstrap required, opened a new log
I20260808 02:15:20.998211 6960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } }
I20260808 02:15:20.998344 6960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:20.998383 6960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f60db7d86c4845bb9239567b329537, State: Initialized, Role: FOLLOWER
I20260808 02:15:20.998497 6960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } }
I20260808 02:15:20.998739 7210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } } }
I20260808 02:15:20.998781 6960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:20.998824 7210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:15:21.000629 7224 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:15:21.000730 7224 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:15:21.001582 6876 master_runner.cc:186] Error getting master registration for 127.19.58.253:40967: OK, instance_id { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" instance_seqno: 1786155320793181 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:15:21.003494 6876 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:15:21.025453 7064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } attempt: 3
Aug 08 02:15:21 dist-test-slave-1l3l krb5kdc[6856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155320, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
I20260808 02:15:21.035892 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:21.025918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34089 (local address 127.0.0.1:50152)
0808 02:15:21.028045 (+ 2127us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:21.028057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:21.028112 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:21.028339 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:21.028341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:21.028555 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:21.028611 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:21.028615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:21.029593 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:21.029594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:21.030340 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:21.030343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:21.030877 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:21.030879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:21.030950 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:21.031467 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:21.031472 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:21.033081 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:21.034936 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:21.034937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:21.034938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:21.035096 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:21.035283 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:21.035284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:21.035285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:21.035348 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:21.035563 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:21.035564 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:21.035607 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 02:15:21.035666 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:15:21.035785 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2101}
I20260808 02:15:21.035909 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:21.025946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.19.58.252:34089)
0808 02:15:21.026016 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:21.026018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:21.028165 (+ 2147us) server_negotiation.cc:407] Connection header received
0808 02:15:21.028191 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:21.028192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:21.028205 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:21.028268 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:21.028713 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:21.029478 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:21.030449 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:21.030744 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:21.030809 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:21.033176 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:21.033178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:21.033178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:21.033185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:21.034858 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:21.035159 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:21.035160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:21.035160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:21.035229 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:21.035399 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:21.035399 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:21.035400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:21.035511 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:21.035565 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:21.035693 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 02:15:21.035786 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 02:15:21.037370 7064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb: Bootstrap starting.
I20260808 02:15:21.038022 7064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:21.038307 7064 log.cc:824] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:21.039042 7064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb: No bootstrap required, opened a new log
I20260808 02:15:21.040480 7064 raft_consensus.cc:348] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } }
I20260808 02:15:21.040591 7064 raft_consensus.cc:374] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:21.040624 7064 raft_consensus.cc:729] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abbc7e2d67e64237b10722e5c306bedb, State: Initialized, Role: FOLLOWER
I20260808 02:15:21.040781 7064 consensus_queue.cc:260] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } }
I20260808 02:15:21.041044 7228 sys_catalog.cc:455] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } } }
I20260808 02:15:21.041096 7064 sys_catalog.cc:564] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:21.041157 7228 sys_catalog.cc:458] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:15:21.044035 7243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:15:21.044143 7243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:15:21.045045 6969 master_runner.cc:263] No leader master found from master abbc7e2d67e64237b10722e5c306bedb
I20260808 02:15:21.045086 6969 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 02:15:21.140482 6901 tablet.cc:2369] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:15:21.187510 7102 tablet.cc:2369] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:15:21.276903 7228 raft_consensus.cc:482] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:21.277021 7228 raft_consensus.cc:504] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } }
I20260808 02:15:21.277246 7228 leader_election.cc:288] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11f60db7d86c4845bb9239567b329537 (127.19.58.254:46773), dc8c1c452e884e54b2094635d04ce7a9 (127.19.58.252:34089)
I20260808 02:15:21.277508 7135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "abbc7e2d67e64237b10722e5c306bedb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc8c1c452e884e54b2094635d04ce7a9" is_pre_election: true
I20260808 02:15:21.277523 6924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "abbc7e2d67e64237b10722e5c306bedb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f60db7d86c4845bb9239567b329537" is_pre_election: true
I20260808 02:15:21.277647 7135 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abbc7e2d67e64237b10722e5c306bedb in term 0.
I20260808 02:15:21.277673 6924 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abbc7e2d67e64237b10722e5c306bedb in term 0.
I20260808 02:15:21.277844 6992 leader_election.cc:302] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abbc7e2d67e64237b10722e5c306bedb, dc8c1c452e884e54b2094635d04ce7a9; no voters:
I20260808 02:15:21.278015 7228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:21.278074 7228 raft_consensus.cc:482] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:21.278090 7228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:21.278811 7228 raft_consensus.cc:504] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } }
I20260808 02:15:21.278949 7228 leader_election.cc:288] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [CANDIDATE]: Term 1 election: Requested vote from peers 11f60db7d86c4845bb9239567b329537 (127.19.58.254:46773), dc8c1c452e884e54b2094635d04ce7a9 (127.19.58.252:34089)
I20260808 02:15:21.279125 7135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "abbc7e2d67e64237b10722e5c306bedb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc8c1c452e884e54b2094635d04ce7a9"
I20260808 02:15:21.279121 6924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "abbc7e2d67e64237b10722e5c306bedb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11f60db7d86c4845bb9239567b329537"
I20260808 02:15:21.279204 6924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:21.279206 7135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:21.279918 7135 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abbc7e2d67e64237b10722e5c306bedb in term 1.
I20260808 02:15:21.280069 6992 leader_election.cc:302] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abbc7e2d67e64237b10722e5c306bedb, dc8c1c452e884e54b2094635d04ce7a9; no voters:
I20260808 02:15:21.280081 6924 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abbc7e2d67e64237b10722e5c306bedb in term 1.
I20260808 02:15:21.280157 7228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:21.280273 7228 raft_consensus.cc:686] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [term 1 LEADER]: Becoming Leader. State: Replica: abbc7e2d67e64237b10722e5c306bedb, State: Running, Role: LEADER
I20260808 02:15:21.280360 7228 consensus_queue.cc:237] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } }
I20260808 02:15:21.280754 7245 sys_catalog.cc:455] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [sys.catalog]: SysCatalogTable state changed. Reason: New leader abbc7e2d67e64237b10722e5c306bedb. Latest consensus state: current_term: 1 leader_uuid: "abbc7e2d67e64237b10722e5c306bedb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } } }
I20260808 02:15:21.280830 7245 sys_catalog.cc:458] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:21.281030 7248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:21.281193 7248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:21.282437 6924 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [term 1 FOLLOWER]: Refusing update from remote peer abbc7e2d67e64237b10722e5c306bedb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:21.282467 7135 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [term 1 FOLLOWER]: Refusing update from remote peer abbc7e2d67e64237b10722e5c306bedb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:21.282717 7228 consensus_queue.cc:1035] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [LEADER]: Connected to new peer: Peer: permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:21.282817 7245 consensus_queue.cc:1035] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:21.283934 7210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abbc7e2d67e64237b10722e5c306bedb. Latest consensus state: current_term: 1 leader_uuid: "abbc7e2d67e64237b10722e5c306bedb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } } }
I20260808 02:15:21.284022 7210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:15:21.284045 7180 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abbc7e2d67e64237b10722e5c306bedb. Latest consensus state: current_term: 1 leader_uuid: "abbc7e2d67e64237b10722e5c306bedb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } } }
I20260808 02:15:21.284132 7180 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:15:21.285156 7228 sys_catalog.cc:455] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "abbc7e2d67e64237b10722e5c306bedb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } } }
I20260808 02:15:21.285238 7228 sys_catalog.cc:458] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:21.285275 7228 sys_catalog.cc:455] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "abbc7e2d67e64237b10722e5c306bedb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } } }
I20260808 02:15:21.285312 7228 sys_catalog.cc:458] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:21.285388 7180 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "abbc7e2d67e64237b10722e5c306bedb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } } }
I20260808 02:15:21.285462 7180 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:15:21.285521 7210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "abbc7e2d67e64237b10722e5c306bedb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f60db7d86c4845bb9239567b329537" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 46773 } } peers { permanent_uuid: "abbc7e2d67e64237b10722e5c306bedb" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 40967 } } peers { permanent_uuid: "dc8c1c452e884e54b2094635d04ce7a9" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34089 } } }
I20260808 02:15:21.285571 7248 catalog_manager.cc:1347] Generated new cluster ID: cad0ed2eb2354720831a5730003b48a9
I20260808 02:15:21.285593 7210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:15:21.285611 7248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:21.285717 7250 mvcc.cc:205] Tried to move back new op lower bound from 7316092195970768896 to 7316092195964899328. Current Snapshot: MvccSnapshot[applied={T|T < 7316092195970768896}]
I20260808 02:15:21.298431 7248 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:15:21.299057 7248 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:21.303148 7248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abbc7e2d67e64237b10722e5c306bedb: Generated new TSK 0
I20260808 02:15:21.303375 7248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:15:21.973203 7193 catalog_manager.cc:1259] Loaded cluster ID: cad0ed2eb2354720831a5730003b48a9
I20260808 02:15:21.973290 7193 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9: loading cluster ID for follower catalog manager: success
I20260808 02:15:21.974107 7193 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9: acquiring CA information for follower catalog manager: success
I20260808 02:15:21.974395 7193 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dc8c1c452e884e54b2094635d04ce7a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:15:22.001176 7224 catalog_manager.cc:1259] Loaded cluster ID: cad0ed2eb2354720831a5730003b48a9
I20260808 02:15:22.001241 7224 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537: loading cluster ID for follower catalog manager: success
I20260808 02:15:22.002058 7224 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537: acquiring CA information for follower catalog manager: success
I20260808 02:15:22.002338 7224 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 11f60db7d86c4845bb9239567b329537: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:15:22.353765 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 6876
I20260808 02:15:22.373871 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 6969
I20260808 02:15:22.380456 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 7073
2026-08-08T02:15:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3084 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:15:22 dist-test-slave-1l3l krb5kdc[7270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:15:22 dist-test-slave-1l3l krb5kdc[7270](info): set up 2 sockets
Aug 08 02:15:22 dist-test-slave-1l3l krb5kdc[7270](info): commencing operation
krb5kdc: starting...
W20260808 02:15:23.494861 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:15:23 dist-test-slave-1l3l krb5kdc[7270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:15:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:15:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:15:23.608528 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:33539
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:41365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:33539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:23.686678 7288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:23.686877 7288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:23.686906 7288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:23.688061 7288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:23.688102 7288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:23.688122 7288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:23.688136 7288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:23.688153 7288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:23.689574 7288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:41365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:33539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:33539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:23.689797 7288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:23.689992 7288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:23.694481 7295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:23.694541 7296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:23.694602 7288 server_base.cc:1034] running on GCE node
W20260808 02:15:23.694509 7299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:23.694885 7288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:23.695149 7288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:23.696295 7288 hybrid_clock.cc:648] HybridClock initialized: now 1786155323696275 us; error 29 us; skew 500 ppm
Aug 08 02:15:23 dist-test-slave-1l3l krb5kdc[7270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155323, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:23.698580 7288 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:15:23.698992 7288 webserver.cc:460] Webserver started at http://127.19.58.254:46883/ using document root <none> and password file <none>
I20260808 02:15:23.699179 7288 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:23.699226 7288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:23.699350 7288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:23.700405 7288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "08ccef2dd91c4a3f8667ec2390b62ea4"
format_stamp: "Formatted at 2026-08-08 02:15:23 on dist-test-slave-1l3l"
server_key: "08bad7a2e04180a9966b89f750f8ba41"
server_key_iv: "69cdcdb089c546f68a554a3728aa1851"
server_key_version: "encryptionkey@0"
I20260808 02:15:23.700726 7288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "08ccef2dd91c4a3f8667ec2390b62ea4"
format_stamp: "Formatted at 2026-08-08 02:15:23 on dist-test-slave-1l3l"
server_key: "08bad7a2e04180a9966b89f750f8ba41"
server_key_iv: "69cdcdb089c546f68a554a3728aa1851"
server_key_version: "encryptionkey@0"
I20260808 02:15:23.701970 7288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:15:23.702618 7306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:23.702752 7288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:15:23.702843 7288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "08ccef2dd91c4a3f8667ec2390b62ea4"
format_stamp: "Formatted at 2026-08-08 02:15:23 on dist-test-slave-1l3l"
server_key: "08bad7a2e04180a9966b89f750f8ba41"
server_key_iv: "69cdcdb089c546f68a554a3728aa1851"
server_key_version: "encryptionkey@0"
I20260808 02:15:23.702929 7288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:23.710011 7288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:23.711941 7288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:23.712067 7288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:23.715852 7288 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:33539
I20260808 02:15:23.715909 7379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:33539 every 8 connection(s)
I20260808 02:15:23.716194 7288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:23.716964 7381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:23.719015 7381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4: Bootstrap starting.
I20260808 02:15:23.719542 7381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:23.719837 7381 log.cc:824] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:23.720497 7381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4: No bootstrap required, opened a new log
I20260808 02:15:23.722018 7381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ccef2dd91c4a3f8667ec2390b62ea4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 33539 } }
I20260808 02:15:23.722143 7381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:23.722174 7381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08ccef2dd91c4a3f8667ec2390b62ea4, State: Initialized, Role: FOLLOWER
I20260808 02:15:23.722311 7381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ccef2dd91c4a3f8667ec2390b62ea4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 33539 } }
I20260808 02:15:23.722386 7381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:23.722421 7381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:23.722463 7381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:23.722782 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 7288
I20260808 02:15:23.722872 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:15:23.723060 19691 external_mini_cluster.cc:1428] Setting key 2290fd88ca6baa83bc41a3dd7ad2906b
I20260808 02:15:23.723053 7381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ccef2dd91c4a3f8667ec2390b62ea4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 33539 } }
I20260808 02:15:23.723137 7381 leader_election.cc:302] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08ccef2dd91c4a3f8667ec2390b62ea4; no voters:
I20260808 02:15:23.723292 7381 leader_election.cc:288] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:15:23.723379 7385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:23.723521 7385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [term 1 LEADER]: Becoming Leader. State: Replica: 08ccef2dd91c4a3f8667ec2390b62ea4, State: Running, Role: LEADER
I20260808 02:15:23.723549 7381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:23.723609 7385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ccef2dd91c4a3f8667ec2390b62ea4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 33539 } }
I20260808 02:15:23.723986 7386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08ccef2dd91c4a3f8667ec2390b62ea4. Latest consensus state: current_term: 1 leader_uuid: "08ccef2dd91c4a3f8667ec2390b62ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ccef2dd91c4a3f8667ec2390b62ea4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 33539 } } }
I20260808 02:15:23.724098 7388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08ccef2dd91c4a3f8667ec2390b62ea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08ccef2dd91c4a3f8667ec2390b62ea4" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 33539 } } }
I20260808 02:15:23.724246 7388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:23.724134 7386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:23.724727 7399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:23.725231 7399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:23.726665 7399 catalog_manager.cc:1347] Generated new cluster ID: 7a35bb8a71c548299757c16cd099a073
I20260808 02:15:23.726716 7399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:15:23 dist-test-slave-1l3l krb5kdc[7270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:23.734438 7389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:23.723503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.19.58.254:33539)
0808 02:15:23.724219 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:23.724229 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:23.724263 (+ 34us) server_negotiation.cc:407] Connection header received
0808 02:15:23.724374 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:23.724381 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:23.724435 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:23.725126 (+ 691us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:23.725377 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:23.726544 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:23.727628 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:23.728243 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:23.728338 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:23.730949 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:23.730954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:23.730954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:23.730967 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:23.732899 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:23.733249 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:23.733252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:23.733253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:23.733369 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:23.733548 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:23.733549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:23.733549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:23.733873 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:23.734020 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:23.734089 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 02:15:23.734223 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":68,"threads_started":1}
I20260808 02:15:23.735494 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:15:23.747900 7399 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:15:23.748358 7399 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:23.751356 7399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08ccef2dd91c4a3f8667ec2390b62ea4: Generated new TSK 0
I20260808 02:15:23.751513 7399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:15:23.789332 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 7288
2026-08-08T02:15:23Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1410 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:15:23 dist-test-slave-1l3l krb5kdc[7418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:15:23 dist-test-slave-1l3l krb5kdc[7418](info): set up 2 sockets
Aug 08 02:15:23 dist-test-slave-1l3l krb5kdc[7418](info): commencing operation
krb5kdc: starting...
W20260808 02:15:24.902109 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:15:24 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:15:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:15:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:15:25.012603 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:39339
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:39339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:25.093323 7440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:25.093516 7440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:25.093537 7440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:25.094623 7440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:25.094663 7440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:25.094676 7440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:25.094688 7440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:25.094703 7440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:25.096159 7440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:39339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:39339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:25.096374 7440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:25.096586 7440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:25.101490 7449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:25.101514 7445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:25.101506 7446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:25.101598 7440 server_base.cc:1034] running on GCE node
I20260808 02:15:25.101887 7440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:25.102164 7440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:25.103314 7440 hybrid_clock.cc:648] HybridClock initialized: now 1786155325103307 us; error 33 us; skew 500 ppm
Aug 08 02:15:25 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:25.105619 7440 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:15:25.106074 7440 webserver.cc:460] Webserver started at http://127.19.58.254:37517/ using document root <none> and password file <none>
I20260808 02:15:25.106289 7440 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:25.106338 7440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:25.106480 7440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:25.107472 7440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "ef8890302dc8418e8083f1f210033001"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "0d513d5eccddd0594f007d812f6d112e"
server_key_iv: "0c21e2d3f3345a976f3845c5eee47a22"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.107834 7440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "ef8890302dc8418e8083f1f210033001"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "0d513d5eccddd0594f007d812f6d112e"
server_key_iv: "0c21e2d3f3345a976f3845c5eee47a22"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.109055 7440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:15:25.109735 7458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.109906 7440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:15:25.109987 7440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "ef8890302dc8418e8083f1f210033001"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "0d513d5eccddd0594f007d812f6d112e"
server_key_iv: "0c21e2d3f3345a976f3845c5eee47a22"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.110052 7440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:25.120925 7440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:25.122506 7440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:25.122632 7440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:25.126538 7440 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:39339
I20260808 02:15:25.126569 7533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:39339 every 8 connection(s)
I20260808 02:15:25.126880 7440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:25.127840 7534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:25.129998 7534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Bootstrap starting.
I20260808 02:15:25.130506 7534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:25.130751 7534 log.cc:824] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:25.131362 7534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: No bootstrap required, opened a new log
I20260808 02:15:25.132979 7534 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:25.133118 7534 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:25.133141 7534 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8890302dc8418e8083f1f210033001, State: Initialized, Role: FOLLOWER
I20260808 02:15:25.133248 7534 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:25.133322 7534 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:25.133350 7534 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:25.133384 7534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:25.134122 7534 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:25.134209 7534 leader_election.cc:302] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef8890302dc8418e8083f1f210033001; no voters:
I20260808 02:15:25.134326 7534 leader_election.cc:288] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:15:25.134373 7537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:25.134532 7537 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 1 LEADER]: Becoming Leader. State: Replica: ef8890302dc8418e8083f1f210033001, State: Running, Role: LEADER
I20260808 02:15:25.134550 7534 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:25.134681 7537 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:25.135046 7538 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef8890302dc8418e8083f1f210033001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } } }
I20260808 02:15:25.135041 7540 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef8890302dc8418e8083f1f210033001. Latest consensus state: current_term: 1 leader_uuid: "ef8890302dc8418e8083f1f210033001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } } }
I20260808 02:15:25.135138 7538 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:25.135141 7540 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:25.135473 7547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:25.135994 7547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:25.136965 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 7440
I20260808 02:15:25.137076 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:15:25.137259 19691 external_mini_cluster.cc:1428] Setting key 277b1774e6f7fa73652a57ab05473b04
I20260808 02:15:25.137455 7547 catalog_manager.cc:1347] Generated new cluster ID: 34f3b1b59ede4aff90dc3bc81104700b
I20260808 02:15:25.137518 7547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:15:25 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:25.148618 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.137906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.19.58.254:39339)
0808 02:15:25.138091 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.138100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.138132 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:15:25.138251 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.138258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.138311 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.138458 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:25.139155 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.140653 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.141642 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.142259 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.142368 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.145100 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:25.145104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:25.145104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:25.145116 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:25.147015 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.147400 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.147403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.147403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.147509 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.147686 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.147687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.147687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.148012 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:25.148139 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.148254 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.148383 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
I20260808 02:15:25.149700 7547 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:15:25.150380 7547 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:25.162194 7547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Generated new TSK 0
I20260808 02:15:25.162372 7547 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:15:25.196710 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39339
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:25.295289 7569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:25.295492 7569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:25.295514 7569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:25.296644 7569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:25.296701 7569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:25.296725 7569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:15:25.298174 7569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:25.298388 7569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:25.298597 7569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:25.303268 7579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:25.303265 7575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:25.303362 7569 server_base.cc:1034] running on GCE node
W20260808 02:15:25.303270 7576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:25.303675 7569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:25.303910 7569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:25.305048 7569 hybrid_clock.cc:648] HybridClock initialized: now 1786155325305036 us; error 30 us; skew 500 ppm
Aug 08 02:15:25 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:25.307400 7569 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:15:25.307821 7569 webserver.cc:460] Webserver started at http://127.19.58.193:43157/ using document root <none> and password file <none>
I20260808 02:15:25.308015 7569 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:25.308063 7569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:25.308189 7569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:25.309136 7569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "089b59afeb6649db814db2198ee396d3"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "294a51e4dc3f54df08575cedb55345c2"
server_key_iv: "72170bd952cab6dcee7daa9362c54bf2"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.309449 7569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "089b59afeb6649db814db2198ee396d3"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "294a51e4dc3f54df08575cedb55345c2"
server_key_iv: "72170bd952cab6dcee7daa9362c54bf2"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.310597 7569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:15:25.311187 7591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.311342 7569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:15:25.311448 7569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "089b59afeb6649db814db2198ee396d3"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "294a51e4dc3f54df08575cedb55345c2"
server_key_iv: "72170bd952cab6dcee7daa9362c54bf2"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.311507 7569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:25.339368 7569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:25.341357 7569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:25.341498 7569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:25.341706 7569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:25.342003 7569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:25.342043 7569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.342073 7569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:25.342090 7569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.347298 7569 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:43709
I20260808 02:15:25.347357 7755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:43709 every 8 connection(s)
I20260808 02:15:25.347632 7569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:25.352447 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 7569
I20260808 02:15:25.352540 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:15:25.352741 19691 external_mini_cluster.cc:1428] Setting key 03607bcef6157ef5227d76c79f796fe8
Aug 08 02:15:25 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:25.356091 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.348277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:50377 (local address 127.19.58.254:39339)
0808 02:15:25.348335 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.348337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.348523 (+ 186us) server_negotiation.cc:407] Connection header received
0808 02:15:25.349000 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.349001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.349014 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.349054 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:25.349912 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.350419 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.351314 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.351513 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.351555 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.353579 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:25.353580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:25.353580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:25.353586 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:25.354731 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.355188 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.355189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.355190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.355253 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.355468 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.355468 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.355469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.355595 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:25.355658 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.355908 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.356001 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 02:15:25.356161 7761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.348232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:50377)
0808 02:15:25.348402 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.348442 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.348909 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.349121 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.349124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.349645 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:25.349828 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.349832 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.350480 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.350480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.351257 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.351258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.351564 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.351565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.351667 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.352091 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:25.352099 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:25.353493 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:25.354802 (+ 1309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:25.354810 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:25.354812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:25.355106 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:25.355322 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:25.355323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:25.355323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:25.355417 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:25.355646 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:25.355648 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:25.355787 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.355869 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.356001 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260808 02:15:25.356595 7756 heartbeater.cc:347] Connected to a master server at 127.19.58.254:39339
I20260808 02:15:25.356676 7756 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:25.356834 7756 heartbeater.cc:511] Master 127.19.58.254:39339 requested a full tablet report, sending...
I20260808 02:15:25.357342 7479 ts_manager.cc:194] Registered new tserver with Master: 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709)
I20260808 02:15:25.358285 7479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:50377
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:15:25.377790 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39339
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:25.455833 7766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:25.456028 7766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:25.456058 7766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:25.457151 7766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:25.457191 7766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:25.457226 7766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:15:25.458598 7766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:25.458817 7766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:25.459020 7766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:25.463609 7775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:25.463644 7766 server_base.cc:1034] running on GCE node
W20260808 02:15:25.463692 7777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:25.463615 7779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:25.463989 7766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:25.464202 7766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:25.465358 7766 hybrid_clock.cc:648] HybridClock initialized: now 1786155325465342 us; error 34 us; skew 500 ppm
Aug 08 02:15:25 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:25.467695 7766 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:15:25.468142 7766 webserver.cc:460] Webserver started at http://127.19.58.194:46693/ using document root <none> and password file <none>
I20260808 02:15:25.468334 7766 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:25.468403 7766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:25.468536 7766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:25.469496 7766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "64238340aabc44258e69378025c835c6"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "baee51fd7bc78aff4b4e2c5fd6bbf4de"
server_key_iv: "257105b51f61973319b8e34801303836"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.469818 7766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "64238340aabc44258e69378025c835c6"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "baee51fd7bc78aff4b4e2c5fd6bbf4de"
server_key_iv: "257105b51f61973319b8e34801303836"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.471001 7766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:15:25.471638 7791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.471802 7766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:15:25.471882 7766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "64238340aabc44258e69378025c835c6"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "baee51fd7bc78aff4b4e2c5fd6bbf4de"
server_key_iv: "257105b51f61973319b8e34801303836"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.471947 7766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:25.480523 7766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:25.482174 7766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:25.482298 7766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:25.482491 7766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:25.482757 7766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:25.482789 7766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.482810 7766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:25.482833 7766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.488308 7766 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:35663
I20260808 02:15:25.488368 7941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:35663 every 8 connection(s)
I20260808 02:15:25.488636 7766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:25.492010 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 7766
I20260808 02:15:25.492110 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:15:25.492295 19691 external_mini_cluster.cc:1428] Setting key 90c47bd751eda0d561640675fc91def4
Aug 08 02:15:25 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:25.497453 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.489270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:42347 (local address 127.19.58.254:39339)
0808 02:15:25.489325 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.489327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.489499 (+ 172us) server_negotiation.cc:407] Connection header received
0808 02:15:25.489952 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.489953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.489961 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.490008 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:25.490841 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.491326 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.492557 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.492798 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.492866 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.494809 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:25.494810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:25.494811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:25.494818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:25.496238 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.496591 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.496592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.496592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.496652 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.496843 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.496844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.496844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.496983 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:25.497060 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.497272 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.497352 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:15:25.497578 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.489225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:42347)
0808 02:15:25.489386 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.489424 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.489862 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.490063 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.490066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.490596 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:25.490770 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.490774 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.491385 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.491385 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.492469 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.492471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.492880 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.492881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.493004 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.493451 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:25.493459 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:25.494726 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:25.496309 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:25.496315 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:25.496317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:25.496523 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:25.496710 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:25.496710 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:25.496710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:25.496795 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:25.497046 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:25.497049 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:25.497149 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.497232 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.497378 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260808 02:15:25.498066 7942 heartbeater.cc:347] Connected to a master server at 127.19.58.254:39339
I20260808 02:15:25.498162 7942 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:25.498320 7942 heartbeater.cc:511] Master 127.19.58.254:39339 requested a full tablet report, sending...
I20260808 02:15:25.498708 7479 ts_manager.cc:194] Registered new tserver with Master: 64238340aabc44258e69378025c835c6 (127.19.58.194:35663)
I20260808 02:15:25.499188 7479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:42347
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:15:25.516311 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39339
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:25.594758 7947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:25.594969 7947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:25.594995 7947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:25.596138 7947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:25.596174 7947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:25.596199 7947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:15:25.597558 7947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:39339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:25.597725 7947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:25.597903 7947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:25.602573 7954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:25.602617 7955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:25.602620 7957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:25.602631 7947 server_base.cc:1034] running on GCE node
I20260808 02:15:25.602962 7947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:25.603164 7947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:25.604292 7947 hybrid_clock.cc:648] HybridClock initialized: now 1786155325604273 us; error 33 us; skew 500 ppm
Aug 08 02:15:25 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:25.606521 7947 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:15:25.606928 7947 webserver.cc:460] Webserver started at http://127.19.58.195:39099/ using document root <none> and password file <none>
I20260808 02:15:25.607126 7947 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:25.607173 7947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:25.607295 7947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:25.608342 7947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "0b41674c945041c988290ec858fe8394"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "8f55e5800f519cb7a2ffb21e615c00bd"
server_key_iv: "db8a4e813db8eac86bcb8c2b481d367e"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.608659 7947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "0b41674c945041c988290ec858fe8394"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "8f55e5800f519cb7a2ffb21e615c00bd"
server_key_iv: "db8a4e813db8eac86bcb8c2b481d367e"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.609835 7947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:15:25.610515 7969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.610646 7947 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:15:25.610718 7947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "0b41674c945041c988290ec858fe8394"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "8f55e5800f519cb7a2ffb21e615c00bd"
server_key_iv: "db8a4e813db8eac86bcb8c2b481d367e"
server_key_version: "encryptionkey@0"
I20260808 02:15:25.610777 7947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:25.625814 7947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:25.627409 7947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:25.627534 7947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:25.627756 7947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:25.628043 7947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:25.628078 7947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.628104 7947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:25.628121 7947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:25.633569 7947 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:45021
I20260808 02:15:25.633605 8129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:45021 every 8 connection(s)
I20260808 02:15:25.633868 7947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
Aug 08 02:15:25 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155325, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:25.640756 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 7947
I20260808 02:15:25.640838 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:15:25.640992 19691 external_mini_cluster.cc:1428] Setting key a57fcfaa257bb69d88d598344b762a97
I20260808 02:15:25.642212 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.634526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:53851 (local address 127.19.58.254:39339)
0808 02:15:25.634566 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.634568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.634751 (+ 183us) server_negotiation.cc:407] Connection header received
0808 02:15:25.635209 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.635210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.635220 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.635282 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:25.636235 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.636754 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.637671 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.637848 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.637891 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.639690 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:25.639692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:25.639692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:25.639699 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:25.640982 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.641341 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.641342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.641342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.641413 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.641665 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.641666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.641666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.641780 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:25.641842 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.642092 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.642150 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:15:25.642349 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.634504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:53851)
0808 02:15:25.634647 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.634685 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.635135 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.635388 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.635392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.635975 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:25.636156 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.636160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.636831 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.636832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.637609 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.637611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.637907 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.637907 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.637997 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.638375 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:25.638383 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:25.639601 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:25.641054 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:25.641062 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:25.641063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:25.641275 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:25.641474 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:25.641474 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:25.641475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:25.641604 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:25.641841 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:25.641847 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:25.641983 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.642069 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.642180 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260808 02:15:25.642742 8130 heartbeater.cc:347] Connected to a master server at 127.19.58.254:39339
I20260808 02:15:25.642824 8130 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:25.642982 8130 heartbeater.cc:511] Master 127.19.58.254:39339 requested a full tablet report, sending...
I20260808 02:15:25.643292 7479 ts_manager.cc:194] Registered new tserver with Master: 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021)
I20260808 02:15:25.643639 7479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:53851
I20260808 02:15:25.652614 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:15:25.659112 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.653871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.19.58.254:39339)
0808 02:15:25.653930 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.653931 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.654012 (+ 81us) server_negotiation.cc:407] Connection header received
0808 02:15:25.654071 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.654072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.654082 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.654134 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:25.654680 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.655159 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.655901 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.656082 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.656135 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.657144 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:25.657145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:25.657146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:25.657152 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:25.658114 (+ 962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.658416 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.658417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.658417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.658469 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:25.658629 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:25.658630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:25.658630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:25.658767 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:25.658851 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.658952 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.659025 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 02:15:25.660559 7479 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:15:25.662079 7479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:15:25.667977 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.664182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35663 (local address 127.0.0.1:52040)
0808 02:15:25.664481 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.664490 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.664542 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.664778 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.664781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.664783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:25.664841 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.664843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.665765 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.665766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.666749 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.666752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.667658 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.667659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.667756 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.667772 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.667846 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.667905 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":47,"threads_started":1}
I20260808 02:15:25.667999 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.664286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52040 (local address 127.19.58.194:35663)
0808 02:15:25.664638 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.664642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.664649 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:15:25.664678 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.664679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.664690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.664752 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:25.664927 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.665675 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.666871 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.667581 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.667649 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.667724 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.667865 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.667918 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":80,"threads_started":1}
I20260808 02:15:25.668162 8145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.664165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:43709 (local address 127.0.0.1:50000)
0808 02:15:25.664291 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.664329 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.664388 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.664899 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.664900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.664903 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:25.664967 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.664969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.666300 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.666301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.667239 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.667241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.667996 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.667997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.668067 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.668071 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.668096 (+ 25us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.668123 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260808 02:15:25.668617 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.664343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50000 (local address 127.19.58.193:43709)
0808 02:15:25.664745 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.664748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.664757 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:15:25.664790 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.664792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.664807 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.665041 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:25.665113 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.666184 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.667310 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.667909 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.668416 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.668477 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.668502 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.668539 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":121,"threads_started":1}
I20260808 02:15:25.668709 7858 tablet_service.cc:1467] Processing CreateTablet for tablet aec93cb0b5ff420ca6ca38b2761bc315 (DEFAULT_TABLE table=test-table [id=e0ea4a282d5d41b89da1e24011fad072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:25.669052 7858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aec93cb0b5ff420ca6ca38b2761bc315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:25.669289 7655 tablet_service.cc:1467] Processing CreateTablet for tablet aec93cb0b5ff420ca6ca38b2761bc315 (DEFAULT_TABLE table=test-table [id=e0ea4a282d5d41b89da1e24011fad072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:25.669606 7655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aec93cb0b5ff420ca6ca38b2761bc315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:25.672011 8144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.664363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:45021 (local address 127.0.0.1:40066)
0808 02:15:25.664475 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.664483 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.664521 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.664866 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.664867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.664869 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:25.664899 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.664900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.666322 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.666324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.667243 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.667246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.671765 (+ 4519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.671766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.671830 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.671835 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.671890 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.671948 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260808 02:15:25.672355 8155 tablet_bootstrap.cc:492] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: Bootstrap starting.
I20260808 02:15:25.672422 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.664562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40066 (local address 127.19.58.195:45021)
0808 02:15:25.664712 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.664716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.664726 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:15:25.664764 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.664766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.664781 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.665018 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:25.665072 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.666184 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.667335 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.668148 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.668210 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.668521 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.672171 (+ 3650us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.672214 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260808 02:15:25.672706 8156 tablet_bootstrap.cc:492] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Bootstrap starting.
I20260808 02:15:25.672960 8033 tablet_service.cc:1467] Processing CreateTablet for tablet aec93cb0b5ff420ca6ca38b2761bc315 (DEFAULT_TABLE table=test-table [id=e0ea4a282d5d41b89da1e24011fad072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:25.673220 8033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aec93cb0b5ff420ca6ca38b2761bc315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:25.673254 8155 tablet_bootstrap.cc:654] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:25.673313 8156 tablet_bootstrap.cc:654] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:25.673861 8155 log.cc:824] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:25.673873 8156 log.cc:824] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:25.674778 8155 tablet_bootstrap.cc:492] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: No bootstrap required, opened a new log
I20260808 02:15:25.674851 8155 ts_tablet_manager.cc:1397] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 02:15:25.674860 8156 tablet_bootstrap.cc:492] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: No bootstrap required, opened a new log
I20260808 02:15:25.674935 8156 ts_tablet_manager.cc:1397] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:15:25.675904 8159 tablet_bootstrap.cc:492] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: Bootstrap starting.
I20260808 02:15:25.676452 8159 tablet_bootstrap.cc:654] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:25.676901 8159 log.cc:824] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:25.676868 8156 raft_consensus.cc:348] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.676934 8155 raft_consensus.cc:348] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.677050 8155 raft_consensus.cc:374] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:25.677050 8156 raft_consensus.cc:374] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:25.677093 8156 raft_consensus.cc:729] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089b59afeb6649db814db2198ee396d3, State: Initialized, Role: FOLLOWER
I20260808 02:15:25.677093 8155 raft_consensus.cc:729] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64238340aabc44258e69378025c835c6, State: Initialized, Role: FOLLOWER
I20260808 02:15:25.677202 8155 consensus_queue.cc:260] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.677202 8156 consensus_queue.cc:260] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.677469 8156 ts_tablet_manager.cc:1428] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 02:15:25.677469 8155 ts_tablet_manager.cc:1428] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:15:25.677496 7942 heartbeater.cc:503] Master 127.19.58.254:39339 was elected leader, sending a full tablet report...
I20260808 02:15:25.677579 7756 heartbeater.cc:503] Master 127.19.58.254:39339 was elected leader, sending a full tablet report...
I20260808 02:15:25.678166 8159 tablet_bootstrap.cc:492] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: No bootstrap required, opened a new log
I20260808 02:15:25.678232 8159 ts_tablet_manager.cc:1397] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:15:25.679668 8159 raft_consensus.cc:348] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.679847 8159 raft_consensus.cc:374] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:25.679889 8159 raft_consensus.cc:729] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b41674c945041c988290ec858fe8394, State: Initialized, Role: FOLLOWER
I20260808 02:15:25.679971 8159 consensus_queue.cc:260] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.680186 8159 ts_tablet_manager.cc:1428] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:15:25.680236 8130 heartbeater.cc:503] Master 127.19.58.254:39339 was elected leader, sending a full tablet report...
W20260808 02:15:25.739125 7943 tablet.cc:2369] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:15:25.848361 7759 tablet.cc:2369] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:15:25.884410 8134 tablet.cc:2369] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:15:25.895988 8163 raft_consensus.cc:482] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:25.896128 8163 raft_consensus.cc:504] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.896533 8163 leader_election.cc:288] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709), 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021)
I20260808 02:15:25.897070 8164 raft_consensus.cc:482] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:25.897177 8164 raft_consensus.cc:504] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.897866 8164 leader_election.cc:288] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709), 64238340aabc44258e69378025c835c6 (127.19.58.194:35663)
I20260808 02:15:25.900221 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.896699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:43709 (local address 127.19.58.194:54793)
0808 02:15:25.896757 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.896770 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.896830 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.897018 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.897019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.897022 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:25.897097 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.897099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.897990 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.897991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.899188 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.899190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.899954 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.899955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.900034 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.900037 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.900079 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.900131 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":14}
I20260808 02:15:25.900341 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.896926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:36087 (local address 127.19.58.195:45021)
0808 02:15:25.896971 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.896972 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.897052 (+ 80us) server_negotiation.cc:407] Connection header received
0808 02:15:25.897102 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.897103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.897112 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.897158 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:25.897396 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.898068 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.899330 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.899937 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.899991 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.899997 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.900252 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.900287 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":24}
I20260808 02:15:25.900228 8166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.896817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:45021 (local address 127.19.58.194:36087)
0808 02:15:25.896993 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.897003 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.897050 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.897205 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.897206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.897208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:25.897282 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.897284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.898143 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.898144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.899199 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.899201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.900025 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.900025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.900084 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.900087 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.900126 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.900153 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":110,"threads_started":1}
I20260808 02:15:25.900616 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.896794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:54793 (local address 127.19.58.193:43709)
0808 02:15:25.896861 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.896863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.896873 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:15:25.896904 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.896905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.896916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.896988 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:25.897162 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.897899 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.899286 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.899851 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.899914 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.899923 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.900510 (+ 587us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.900557 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:15:25.900899 8061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aec93cb0b5ff420ca6ca38b2761bc315" candidate_uuid: "64238340aabc44258e69378025c835c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b41674c945041c988290ec858fe8394" is_pre_election: true
I20260808 02:15:25.900884 7691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aec93cb0b5ff420ca6ca38b2761bc315" candidate_uuid: "64238340aabc44258e69378025c835c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089b59afeb6649db814db2198ee396d3" is_pre_election: true
I20260808 02:15:25.900983 8061 raft_consensus.cc:2454] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64238340aabc44258e69378025c835c6 in term 0.
I20260808 02:15:25.901012 7691 raft_consensus.cc:2454] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64238340aabc44258e69378025c835c6 in term 0.
I20260808 02:15:25.901178 7792 leader_election.cc:302] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b41674c945041c988290ec858fe8394, 64238340aabc44258e69378025c835c6; no voters:
I20260808 02:15:25.901345 8163 raft_consensus.cc:2781] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:25.901383 8163 raft_consensus.cc:482] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:25.901388 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.897623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:43709 (local address 127.19.58.195:46765)
0808 02:15:25.897896 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.897906 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.897952 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.898114 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.898115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.898116 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:25.898170 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.898171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.899099 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.899100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.900259 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.900262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.901185 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.901186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.901258 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.901262 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.901291 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.901326 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":1}
I20260808 02:15:25.901410 8163 raft_consensus.cc:3037] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:25.901530 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.897687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:53525 (local address 127.19.58.194:35663)
0808 02:15:25.897721 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.897723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.898034 (+ 311us) server_negotiation.cc:407] Connection header received
0808 02:15:25.898124 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.898125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.898133 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.898161 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:25.898404 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.899098 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.900698 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.901191 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.901230 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.901237 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.901441 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.901475 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:15:25.901676 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.897733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:46765 (local address 127.19.58.193:43709)
0808 02:15:25.897855 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.897857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.897999 (+ 142us) server_negotiation.cc:407] Connection header received
0808 02:15:25.898024 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.898025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.898034 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.898226 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:25.898270 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.899005 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.900357 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.901047 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.901549 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.901558 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.901576 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.901608 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":56,"threads_started":1}
I20260808 02:15:25.901737 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.897625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35663 (local address 127.19.58.195:53525)
0808 02:15:25.897981 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.897989 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:25.898057 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:25.898213 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:25.898213 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:25.898214 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:25.898272 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.898273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.899296 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.899296 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.900609 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:25.900612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.901288 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.901288 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:25.901362 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.901365 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:15:25.901633 (+ 268us) client_negotiation.cc:240] Negotiation successful
0808 02:15:25.901670 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":145,"thread_start_us":102,"threads_started":1}
I20260808 02:15:25.901826 7691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aec93cb0b5ff420ca6ca38b2761bc315" candidate_uuid: "0b41674c945041c988290ec858fe8394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089b59afeb6649db814db2198ee396d3" is_pre_election: true
I20260808 02:15:25.901897 7691 raft_consensus.cc:2454] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b41674c945041c988290ec858fe8394 in term 0.
I20260808 02:15:25.902077 7971 leader_election.cc:302] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 089b59afeb6649db814db2198ee396d3, 0b41674c945041c988290ec858fe8394; no voters:
I20260808 02:15:25.902112 7885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aec93cb0b5ff420ca6ca38b2761bc315" candidate_uuid: "0b41674c945041c988290ec858fe8394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64238340aabc44258e69378025c835c6" is_pre_election: true
I20260808 02:15:25.902204 8164 raft_consensus.cc:2781] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:25.902251 8164 raft_consensus.cc:482] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:25.902279 8164 raft_consensus.cc:3037] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:25.902360 8163 raft_consensus.cc:504] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.902500 8163 leader_election.cc:288] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [CANDIDATE]: Term 1 election: Requested vote from peers 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709), 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021)
I20260808 02:15:25.902580 7885 raft_consensus.cc:2379] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0b41674c945041c988290ec858fe8394 in current term 1: Already voted for candidate 64238340aabc44258e69378025c835c6 in this term.
I20260808 02:15:25.902679 7691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aec93cb0b5ff420ca6ca38b2761bc315" candidate_uuid: "64238340aabc44258e69378025c835c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089b59afeb6649db814db2198ee396d3"
I20260808 02:15:25.902752 7691 raft_consensus.cc:3037] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:25.902794 8061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aec93cb0b5ff420ca6ca38b2761bc315" candidate_uuid: "64238340aabc44258e69378025c835c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b41674c945041c988290ec858fe8394"
I20260808 02:15:25.903090 8164 raft_consensus.cc:504] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.903219 8061 raft_consensus.cc:2379] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 64238340aabc44258e69378025c835c6 in current term 1: Already voted for candidate 0b41674c945041c988290ec858fe8394 in this term.
I20260808 02:15:25.903368 7885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aec93cb0b5ff420ca6ca38b2761bc315" candidate_uuid: "0b41674c945041c988290ec858fe8394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64238340aabc44258e69378025c835c6"
I20260808 02:15:25.903441 7885 raft_consensus.cc:2379] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0b41674c945041c988290ec858fe8394 in current term 1: Already voted for candidate 64238340aabc44258e69378025c835c6 in this term.
I20260808 02:15:25.903432 7690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aec93cb0b5ff420ca6ca38b2761bc315" candidate_uuid: "0b41674c945041c988290ec858fe8394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089b59afeb6649db814db2198ee396d3"
I20260808 02:15:25.903622 7691 raft_consensus.cc:2454] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64238340aabc44258e69378025c835c6 in term 1.
I20260808 02:15:25.903231 8164 leader_election.cc:288] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [CANDIDATE]: Term 1 election: Requested vote from peers 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709), 64238340aabc44258e69378025c835c6 (127.19.58.194:35663)
I20260808 02:15:25.903687 7971 leader_election.cc:302] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b41674c945041c988290ec858fe8394; no voters: 089b59afeb6649db814db2198ee396d3, 64238340aabc44258e69378025c835c6
I20260808 02:15:25.903882 7792 leader_election.cc:302] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 089b59afeb6649db814db2198ee396d3, 64238340aabc44258e69378025c835c6; no voters: 0b41674c945041c988290ec858fe8394
I20260808 02:15:25.903963 8163 raft_consensus.cc:2781] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:25.903991 8164 raft_consensus.cc:2726] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 02:15:25.904134 8163 raft_consensus.cc:686] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 1 LEADER]: Becoming Leader. State: Replica: 64238340aabc44258e69378025c835c6, State: Running, Role: LEADER
I20260808 02:15:25.904218 8163 consensus_queue.cc:237] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:25.904956 7478 catalog_manager.cc:5495] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64238340aabc44258e69378025c835c6 (127.19.58.194). New cstate: current_term: 1 leader_uuid: "64238340aabc44258e69378025c835c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } health_report { overall_health: HEALTHY } } }
I20260808 02:15:25.947450 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.945387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.19.58.194:35663)
0808 02:15:25.945434 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:25.945436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:25.945449 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:15:25.945500 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:25.945501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:25.945518 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:25.945570 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:15:25.945711 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.946171 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.946847 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:25.947026 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:25.947071 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:25.947162 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:15:25.947255 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:15:25.947325 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:25.947340 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 02:15:25.947359 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 02:15:25.949177 8061 raft_consensus.cc:1261] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 1 FOLLOWER]: Refusing update from remote peer 64238340aabc44258e69378025c835c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:25.949184 7691 raft_consensus.cc:1261] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Refusing update from remote peer 64238340aabc44258e69378025c835c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:25.949431 8163 consensus_queue.cc:1035] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:25.949573 8169 consensus_queue.cc:1035] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:25.951897 8175 mvcc.cc:205] Tried to move back new op lower bound from 7316092215085297664 to 7316092214904086528. Current Snapshot: MvccSnapshot[applied={T|T < 7316092215085297664}]
I20260808 02:15:25.952710 8173 mvcc.cc:205] Tried to move back new op lower bound from 7316092215085297664 to 7316092214904086528. Current Snapshot: MvccSnapshot[applied={T|T < 7316092215085297664 or (T in {7316092215085297664})}]
I20260808 02:15:25.953898 7478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:25.953977 7478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:25.954650 7478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Sending DeleteTablet for 3 replicas of tablet aec93cb0b5ff420ca6ca38b2761bc315
I20260808 02:15:25.954982 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 7569
I20260808 02:15:25.955044 7655 tablet_service.cc:1514] Processing DeleteTablet for tablet aec93cb0b5ff420ca6ca38b2761bc315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:25 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:50000
I20260808 02:15:25.955054 8033 tablet_service.cc:1514] Processing DeleteTablet for tablet aec93cb0b5ff420ca6ca38b2761bc315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:25 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:40066
I20260808 02:15:25.955505 7858 tablet_service.cc:1514] Processing DeleteTablet for tablet aec93cb0b5ff420ca6ca38b2761bc315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:25 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:52040
I20260808 02:15:25.955801 8185 tablet_replica.cc:330] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: stopping tablet replica
I20260808 02:15:25.955924 8185 raft_consensus.cc:2229] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:25.956037 8185 raft_consensus.cc:2258] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:25.960657 8185 ts_tablet_manager.cc:1905] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:25.962637 8185 ts_tablet_manager.cc:1918] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:25.962719 8185 log.cc:1196] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/aec93cb0b5ff420ca6ca38b2761bc315
I20260808 02:15:25.963075 8185 ts_tablet_manager.cc:1939] T aec93cb0b5ff420ca6ca38b2761bc315 P 0b41674c945041c988290ec858fe8394: Deleting consensus metadata
I20260808 02:15:25.964012 8186 tablet_replica.cc:330] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: stopping tablet replica
I20260808 02:15:25.964035 7459 catalog_manager.cc:4841] TS 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021): tablet aec93cb0b5ff420ca6ca38b2761bc315 (table test-table [id=e0ea4a282d5d41b89da1e24011fad072]) successfully deleted
I20260808 02:15:25.964109 8186 raft_consensus.cc:2229] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:15:25.964188 8186 raft_consensus.cc:2258] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:25.965154 8186 ts_tablet_manager.cc:1905] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:25.966674 8186 ts_tablet_manager.cc:1918] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:25.966770 8186 log.cc:1196] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/aec93cb0b5ff420ca6ca38b2761bc315
I20260808 02:15:25.967079 8186 ts_tablet_manager.cc:1939] T aec93cb0b5ff420ca6ca38b2761bc315 P 64238340aabc44258e69378025c835c6: Deleting consensus metadata
I20260808 02:15:25.967465 7461 catalog_manager.cc:4841] TS 64238340aabc44258e69378025c835c6 (127.19.58.194:35663): tablet aec93cb0b5ff420ca6ca38b2761bc315 (table test-table [id=e0ea4a282d5d41b89da1e24011fad072]) successfully deleted
W20260808 02:15:25.968778 7459 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:43709 (error 108)
I20260808 02:15:25.969094 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 7766
I20260808 02:15:25.969389 8144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:25.969254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:43709 (local address 127.0.0.1:50008)
0808 02:15:25.969309 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:25.969353 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:43709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 02:15:25.969534 7459 catalog_manager.cc:4584] TS 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aec93cb0b5ff420ca6ca38b2761bc315: Network error: Client connection negotiation failed: client connection to 127.19.58.193:43709: connect: Connection refused (error 111)
I20260808 02:15:25.976974 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 7947
I20260808 02:15:25.985350 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 7440
I20260808 02:15:25.993265 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:39339
--webserver_interface=127.19.58.254
--webserver_port=37517
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:39339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:26.073601 8188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:26.073810 8188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:26.073832 8188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:26.074985 8188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:26.075026 8188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:26.075040 8188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:26.075052 8188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:26.075062 8188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:26.076555 8188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:39339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:39339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:26.076776 8188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:26.077004 8188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:26.086323 8198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.086313 8199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.086294 8201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:26.086448 8188 server_base.cc:1034] running on GCE node
I20260808 02:15:26.086792 8188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:26.087083 8188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:26.088217 8188 hybrid_clock.cc:648] HybridClock initialized: now 1786155326088201 us; error 26 us; skew 500 ppm
Aug 08 02:15:26 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:26.090603 8188 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:15:26.091006 8188 webserver.cc:460] Webserver started at http://127.19.58.254:37517/ using document root <none> and password file <none>
I20260808 02:15:26.091212 8188 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:26.091277 8188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:26.092626 8188 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:26.093200 8210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.093348 8188 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:15:26.093458 8188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "ef8890302dc8418e8083f1f210033001"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "0d513d5eccddd0594f007d812f6d112e"
server_key_iv: "0c21e2d3f3345a976f3845c5eee47a22"
server_key_version: "encryptionkey@0"
I20260808 02:15:26.093814 8188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:26.113507 8188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:26.115180 8188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:26.115327 8188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:26.119234 8188 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:39339
I20260808 02:15:26.119277 8280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:39339 every 8 connection(s)
I20260808 02:15:26.119665 8188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:26.121404 8281 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:15:26.122066 8281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Bootstrap starting.
I20260808 02:15:26.128260 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 8188
I20260808 02:15:26.128530 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:43709
--local_ip_for_outbound_sockets=127.19.58.193
--tserver_master_addrs=127.19.58.254:39339
--webserver_port=43157
--webserver_interface=127.19.58.193
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:26.131016 8281 log.cc:824] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:26.132843 8281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:26.133116 8281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Bootstrap complete.
I20260808 02:15:26.135221 8281 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:26.135452 8281 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8890302dc8418e8083f1f210033001, State: Initialized, Role: FOLLOWER
I20260808 02:15:26.135604 8281 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:26.135664 8281 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:26.135727 8281 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:26.135778 8281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:15:26.136754 8281 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:26.136839 8281 leader_election.cc:302] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef8890302dc8418e8083f1f210033001; no voters:
I20260808 02:15:26.136991 8281 leader_election.cc:288] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:15:26.137054 8285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:15:26.137173 8285 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 2 LEADER]: Becoming Leader. State: Replica: ef8890302dc8418e8083f1f210033001, State: Running, Role: LEADER
I20260808 02:15:26.137254 8281 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:26.137329 8285 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:26.137648 8288 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ef8890302dc8418e8083f1f210033001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } } }
I20260808 02:15:26.137652 8285 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef8890302dc8418e8083f1f210033001. Latest consensus state: current_term: 2 leader_uuid: "ef8890302dc8418e8083f1f210033001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } } }
I20260808 02:15:26.137744 8288 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:26.137753 8285 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:26.138399 8296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:26.139201 8296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:26.139297 8296 catalog_manager.cc:1259] Loaded cluster ID: 34f3b1b59ede4aff90dc3bc81104700b
I20260808 02:15:26.139365 8296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:15:26.139648 8296 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 02:15:26.142340 8296 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 02:15:26.144624 8296 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:26.144817 8296 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Loaded TSK: 0
I20260808 02:15:26.145020 8296 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:15:26.225939 8283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:26.226138 8283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:26.226161 8283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:26.227283 8283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:26.227324 8283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:26.227355 8283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:15:26.228758 8283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:43709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=43157
--tserver_master_addrs=127.19.58.254:39339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:26.228958 8283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:26.229172 8283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:26.234994 8319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.235001 8318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.235001 8321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:26.235780 8283 server_base.cc:1034] running on GCE node
I20260808 02:15:26.235963 8283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:26.236219 8283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:26.237422 8283 hybrid_clock.cc:648] HybridClock initialized: now 1786155326237404 us; error 42 us; skew 500 ppm
Aug 08 02:15:26 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:26.239975 8283 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:15:26.240437 8283 webserver.cc:460] Webserver started at http://127.19.58.193:43157/ using document root <none> and password file <none>
I20260808 02:15:26.240629 8283 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:26.240693 8283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:26.242080 8283 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:26.242776 8331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.242955 8283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:26.243054 8283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "089b59afeb6649db814db2198ee396d3"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "294a51e4dc3f54df08575cedb55345c2"
server_key_iv: "72170bd952cab6dcee7daa9362c54bf2"
server_key_version: "encryptionkey@0"
I20260808 02:15:26.243453 8283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:26.264834 8283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:26.267365 8283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:26.267530 8283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:26.267833 8283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:26.268379 8341 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:15:26.269784 8283 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:15:26.269850 8283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 02:15:26.269883 8283 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:15:26.270787 8283 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:15:26.270843 8283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 02:15:26.270890 8341 tablet_bootstrap.cc:492] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Bootstrap starting.
I20260808 02:15:26.277727 8283 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:43709
I20260808 02:15:26.277937 8482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:43709 every 8 connection(s)
I20260808 02:15:26.278223 8283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:26.281442 8341 log.cc:824] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:26.283641 8341 tablet_bootstrap.cc:492] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:26.283967 8341 tablet_bootstrap.cc:492] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Bootstrap complete.
I20260808 02:15:26.284315 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 8283
I20260808 02:15:26.284605 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:35663
--local_ip_for_outbound_sockets=127.19.58.194
--tserver_master_addrs=127.19.58.254:39339
--webserver_port=46693
--webserver_interface=127.19.58.194
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:26.285045 8341 ts_tablet_manager.cc:1397] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260808 02:15:26.286363 8341 raft_consensus.cc:348] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:26.286662 8341 raft_consensus.cc:729] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089b59afeb6649db814db2198ee396d3, State: Initialized, Role: FOLLOWER
I20260808 02:15:26.286818 8341 consensus_queue.cc:260] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } }
I20260808 02:15:26.287082 8341 ts_tablet_manager.cc:1428] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Time spent starting tablet: real 0.002s user 0.001s sys 0.003s
Aug 08 02:15:26 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:26.293322 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.279009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:48573 (local address 127.19.58.254:39339)
0808 02:15:26.283850 (+ 4841us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.283859 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.283901 (+ 42us) server_negotiation.cc:407] Connection header received
0808 02:15:26.284032 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.284039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.284097 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.285094 (+ 997us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:26.285185 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.286556 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.287732 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.288107 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.288201 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.290237 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:26.290241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:26.290241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:26.290252 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:26.291785 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.292145 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.292148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.292149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.292233 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.292463 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.292463 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.292464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.292672 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:26.292738 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.293028 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.293148 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4789,"thread_start_us":81,"threads_started":1}
I20260808 02:15:26.293322 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.278904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:48573)
0808 02:15:26.279086 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.279128 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:26.279769 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:26.284325 (+ 4556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:26.284329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:26.284671 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:26.284957 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.284964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.286669 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.286670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.287618 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.287621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.288189 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.288190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.288316 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.288795 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:26.288803 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:26.290140 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:26.291850 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.291857 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.291859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.292072 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.292300 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.292301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.292301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.292406 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.292745 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:26.292752 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:26.292879 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 02:15:26.292965 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:15:26.293127 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260808 02:15:26.293922 8483 heartbeater.cc:347] Connected to a master server at 127.19.58.254:39339
I20260808 02:15:26.294014 8483 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:26.294178 8483 heartbeater.cc:511] Master 127.19.58.254:39339 requested a full tablet report, sending...
I20260808 02:15:26.294811 8231 ts_manager.cc:194] Registered new tserver with Master: 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709)
I20260808 02:15:26.295799 8231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:48573
I20260808 02:15:26.296725 8483 heartbeater.cc:503] Master 127.19.58.254:39339 was elected leader, sending a full tablet report...
Aug 08 02:15:26 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:15:26.302801 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.295649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.19.58.193:43709)
0808 02:15:26.295822 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.295826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.295838 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:15:26.295871 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.295873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.295887 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.295975 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:26.296418 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.297274 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.297983 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.298263 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.298318 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.300288 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:26.300295 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:26.300295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:26.300301 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:26.301711 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.302116 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.302118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.302118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.302164 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.302330 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.302330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.302330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.302466 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:26.302521 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.302674 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.302742 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":85,"threads_started":1}
I20260808 02:15:26.302860 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.295406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:43709 (local address 127.0.0.1:50010)
0808 02:15:26.295567 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.295604 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:26.295690 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:26.295992 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:26.295997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:26.296300 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:26.296344 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.296346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.297361 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.297361 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.297912 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.297914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.298327 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.298328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.298402 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.298804 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:26.298808 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:26.300189 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:26.301789 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.301795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.301797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.302047 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.302216 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.302217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.302217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.302279 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.302517 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:26.302522 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:26.302571 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 02:15:26.302652 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 02:15:26.302759 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260808 02:15:26.303263 8396 tablet_service.cc:1514] Processing DeleteTablet for tablet aec93cb0b5ff420ca6ca38b2761bc315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:25 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:50010
I20260808 02:15:26.303853 8497 tablet_replica.cc:330] stopping tablet replica
I20260808 02:15:26.303957 8497 raft_consensus.cc:2229] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:26.304024 8497 raft_consensus.cc:2258] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:26.304304 8497 ts_tablet_manager.cc:1905] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:26.306308 8497 ts_tablet_manager.cc:1918] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:26.306381 8497 log.cc:1196] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/aec93cb0b5ff420ca6ca38b2761bc315
I20260808 02:15:26.306589 8497 ts_tablet_manager.cc:1939] T aec93cb0b5ff420ca6ca38b2761bc315 P 089b59afeb6649db814db2198ee396d3: Deleting consensus metadata
I20260808 02:15:26.307042 8212 catalog_manager.cc:4841] TS 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709): tablet aec93cb0b5ff420ca6ca38b2761bc315 (table test-table [id=e0ea4a282d5d41b89da1e24011fad072]) successfully deleted
W20260808 02:15:26.367518 8490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:26.367736 8490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:26.367761 8490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:26.368913 8490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:26.368964 8490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:26.368989 8490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:15:26.370361 8490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:35663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=46693
--tserver_master_addrs=127.19.58.254:39339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:26.370560 8490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:26.370769 8490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:26.375571 8505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.375628 8506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:26.375592 8490 server_base.cc:1034] running on GCE node
W20260808 02:15:26.375551 8511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:26.376019 8490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:26.376236 8490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:26.377372 8490 hybrid_clock.cc:648] HybridClock initialized: now 1786155326377396 us; error 66 us; skew 500 ppm
Aug 08 02:15:26 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:26.379876 8490 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:15:26.380282 8490 webserver.cc:460] Webserver started at http://127.19.58.194:46693/ using document root <none> and password file <none>
I20260808 02:15:26.380499 8490 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:26.380565 8490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:26.381948 8490 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:26.382457 8525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.382606 8490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:15:26.382684 8490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "64238340aabc44258e69378025c835c6"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "baee51fd7bc78aff4b4e2c5fd6bbf4de"
server_key_iv: "257105b51f61973319b8e34801303836"
server_key_version: "encryptionkey@0"
I20260808 02:15:26.383016 8490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:26.408407 8490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:26.410085 8490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:26.410220 8490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:26.410423 8490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:26.410703 8490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:26.410746 8490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.410776 8490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:26.410801 8490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.416402 8490 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:35663
I20260808 02:15:26.416460 8676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:35663 every 8 connection(s)
I20260808 02:15:26.416755 8490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:26.421440 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 8490
I20260808 02:15:26.421705 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:45021
--local_ip_for_outbound_sockets=127.19.58.195
--tserver_master_addrs=127.19.58.254:39339
--webserver_port=39099
--webserver_interface=127.19.58.195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 02:15:26 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:26.425465 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.417412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:36101 (local address 127.19.58.254:39339)
0808 02:15:26.417470 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.417472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.417655 (+ 183us) server_negotiation.cc:407] Connection header received
0808 02:15:26.418113 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.418114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.418123 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.418176 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:26.419015 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.419505 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.420506 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.420707 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.420758 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.422874 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:26.422876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:26.422876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:26.422886 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:26.424249 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.424623 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.424624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.424624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.424669 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.424924 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.424924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.424925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.425016 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:26.425059 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.425334 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.425400 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 02:15:26.425688 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.417387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:36101)
0808 02:15:26.417540 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.417579 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:26.418040 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:26.418227 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:26.418230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:26.418763 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:26.418945 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.418950 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.419571 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.419572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.420432 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.420435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.420765 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.420765 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.420855 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.421251 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:26.421259 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:26.422764 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:26.424321 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.424328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.424330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.424564 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.424732 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.424733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.424733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.424858 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.425065 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:26.425071 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:26.425213 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 02:15:26.425329 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 02:15:26.425476 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260808 02:15:26.426186 8678 heartbeater.cc:347] Connected to a master server at 127.19.58.254:39339
I20260808 02:15:26.426285 8678 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:26.426476 8678 heartbeater.cc:511] Master 127.19.58.254:39339 requested a full tablet report, sending...
I20260808 02:15:26.426857 8231 ts_manager.cc:194] Registered new tserver with Master: 64238340aabc44258e69378025c835c6 (127.19.58.194:35663)
I20260808 02:15:26.427213 8231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:36101
W20260808 02:15:26.517009 8684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:26.517195 8684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:26.517217 8684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:26.518297 8684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:26.518347 8684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:26.518378 8684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:15:26.519824 8684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:45021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=39099
--tserver_master_addrs=127.19.58.254:39339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:26.520030 8684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:26.520232 8684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:26.524817 8690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.524859 8693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.524899 8691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:26.525051 8684 server_base.cc:1034] running on GCE node
I20260808 02:15:26.525246 8684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:26.525461 8684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:26.526587 8684 hybrid_clock.cc:648] HybridClock initialized: now 1786155326526579 us; error 26 us; skew 500 ppm
Aug 08 02:15:26 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:26.528947 8684 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:15:26.529361 8684 webserver.cc:460] Webserver started at http://127.19.58.195:39099/ using document root <none> and password file <none>
I20260808 02:15:26.529560 8684 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:26.529621 8684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:26.530853 8684 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:15:26.531368 8706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.531525 8684 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:15:26.531603 8684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "0b41674c945041c988290ec858fe8394"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "8f55e5800f519cb7a2ffb21e615c00bd"
server_key_iv: "db8a4e813db8eac86bcb8c2b481d367e"
server_key_version: "encryptionkey@0"
I20260808 02:15:26.531939 8684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:26.546906 8684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:26.548978 8684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:26.549110 8684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:26.549309 8684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:26.549587 8684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:26.549619 8684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.549646 8684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:26.549670 8684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.555335 8684 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:45021
I20260808 02:15:26.555374 8869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:45021 every 8 connection(s)
I20260808 02:15:26.555680 8684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:15:26.560737 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 8684
Aug 08 02:15:26 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:26.564277 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.556349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:57377 (local address 127.19.58.254:39339)
0808 02:15:26.556401 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.556403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.556582 (+ 179us) server_negotiation.cc:407] Connection header received
0808 02:15:26.557041 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.557042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.557051 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.557102 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:26.557916 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.558402 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.559393 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.559584 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.559637 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.561753 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:26.561755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:26.561756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:26.561763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:26.563018 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.563375 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.563376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.563377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.563432 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.563640 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.563641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.563641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.563794 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:26.563855 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.564112 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.564189 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:15:26.564528 8877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.556320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:57377)
0808 02:15:26.556469 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.556507 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:26.556964 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:26.557152 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:26.557155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:26.557673 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:26.557845 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.557849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.558495 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.558497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.559274 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.559277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.559651 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.559652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.559765 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.560215 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:26.560223 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:26.561663 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:26.563091 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.563098 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.563100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.563300 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.563484 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:26.563485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:26.563485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:26.563584 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:26.563854 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:26.563861 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:26.563987 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 02:15:26.564081 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 02:15:26.564225 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260808 02:15:26.564972 8870 heartbeater.cc:347] Connected to a master server at 127.19.58.254:39339
I20260808 02:15:26.565070 8870 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:26.565250 8870 heartbeater.cc:511] Master 127.19.58.254:39339 requested a full tablet report, sending...
I20260808 02:15:26.565691 8231 ts_manager.cc:194] Registered new tserver with Master: 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021)
I20260808 02:15:26.566148 8231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:57377
I20260808 02:15:26.566835 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.561121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59690 (local address 127.19.58.254:39339)
0808 02:15:26.561252 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.561254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.561262 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:15:26.561296 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.561297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.561307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.561357 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:26.561873 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.562361 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.563050 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.563237 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.563289 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.564415 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:26.564416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:26.564416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:26.564423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:26.565813 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.566170 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.566171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.566171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.566215 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:26.566389 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:26.566389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:26.566390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:26.566485 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:26.566636 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.566675 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.566751 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260808 02:15:26.567433 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:15:26.569591 8879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.567722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.19.58.254:39339)
0808 02:15:26.567753 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.567754 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.567837 (+ 83us) server_negotiation.cc:407] Connection header received
0808 02:15:26.567891 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.567896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.567922 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.567978 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:15:26.568042 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.568457 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.569092 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.569259 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.569298 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.569391 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:15:26.569471 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:15:26.569520 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.569535 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.569550 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260808 02:15:26.569823 8231 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:15:26.571401 8231 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:15:26.573464 8396 tablet_service.cc:1467] Processing CreateTablet for tablet e5b88ff0ab734857bb5ac7aeef98cfbd (DEFAULT_TABLE table=test-table [id=58520b14285344f8a2106e9aa9abb9c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:26.573670 8396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b88ff0ab734857bb5ac7aeef98cfbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:26.575022 8341 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Bootstrap starting.
I20260808 02:15:26.575349 8341 tablet_bootstrap.cc:654] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:26.575906 8341 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: No bootstrap required, opened a new log
I20260808 02:15:26.575961 8341 ts_tablet_manager.cc:1397] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:15:26.576279 8341 raft_consensus.cc:348] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:26.576342 8341 raft_consensus.cc:374] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:26.576396 8341 raft_consensus.cc:729] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089b59afeb6649db814db2198ee396d3, State: Initialized, Role: FOLLOWER
I20260808 02:15:26.576440 8341 consensus_queue.cc:260] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:26.576536 8341 ts_tablet_manager.cc:1428] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 02:15:26.576972 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.573048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35663 (local address 127.0.0.1:52052)
0808 02:15:26.573089 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.573098 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:26.573132 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:26.573497 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:26.573498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:26.573500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:26.573549 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.573550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.574809 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.574810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.575846 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.575849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.576761 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.576762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.576832 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.576835 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:15:26.576864 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 02:15:26.576904 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 02:15:26.576990 8883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.573127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.19.58.195:45021)
0808 02:15:26.573350 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.573354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.573362 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:15:26.573420 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.573421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.573433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.573504 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:26.573657 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.574822 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.575917 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.576604 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.576679 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.576738 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.576861 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.576916 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
I20260808 02:15:26.577288 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.573138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.19.58.194:35663)
0808 02:15:26.573314 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.573319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.573330 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:15:26.573370 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.573372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.573387 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.573485 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:26.573632 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.574701 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.575939 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.576674 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.577070 (+ 396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.577140 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.577166 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.577207 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260808 02:15:26.577374 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.573048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:45021 (local address 127.0.0.1:40074)
0808 02:15:26.573249 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.573258 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:26.573294 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:26.573558 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:26.573558 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:26.573560 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:26.573590 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.573591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.574911 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.574912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.575830 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.575833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.576680 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.576682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.576743 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.576762 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 02:15:26.577276 (+ 514us) client_negotiation.cc:240] Negotiation successful
0808 02:15:26.577310 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":11,"thread_start_us":85,"threads_started":1}
I20260808 02:15:26.577898 8588 tablet_service.cc:1467] Processing CreateTablet for tablet e5b88ff0ab734857bb5ac7aeef98cfbd (DEFAULT_TABLE table=test-table [id=58520b14285344f8a2106e9aa9abb9c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:26.578007 8781 tablet_service.cc:1467] Processing CreateTablet for tablet e5b88ff0ab734857bb5ac7aeef98cfbd (DEFAULT_TABLE table=test-table [id=58520b14285344f8a2106e9aa9abb9c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:26.578246 8588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b88ff0ab734857bb5ac7aeef98cfbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:26.578270 8781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b88ff0ab734857bb5ac7aeef98cfbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:26.581601 8888 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Bootstrap starting.
I20260808 02:15:26.581795 8889 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Bootstrap starting.
I20260808 02:15:26.582360 8889 tablet_bootstrap.cc:654] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:26.582806 8889 log.cc:824] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:26.583421 8889 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: No bootstrap required, opened a new log
I20260808 02:15:26.583427 8888 tablet_bootstrap.cc:654] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:26.583478 8889 ts_tablet_manager.cc:1397] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:15:26.583885 8888 log.cc:824] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:26.584474 8888 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: No bootstrap required, opened a new log
I20260808 02:15:26.584532 8888 ts_tablet_manager.cc:1397] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 02:15:26.584889 8889 raft_consensus.cc:348] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:26.585018 8889 raft_consensus.cc:374] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:26.585038 8889 raft_consensus.cc:729] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b41674c945041c988290ec858fe8394, State: Initialized, Role: FOLLOWER
I20260808 02:15:26.585135 8889 consensus_queue.cc:260] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:26.585381 8889 ts_tablet_manager.cc:1428] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:15:26.585415 8870 heartbeater.cc:503] Master 127.19.58.254:39339 was elected leader, sending a full tablet report...
I20260808 02:15:26.586123 8888 raft_consensus.cc:348] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:26.586273 8888 raft_consensus.cc:374] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:26.586318 8888 raft_consensus.cc:729] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64238340aabc44258e69378025c835c6, State: Initialized, Role: FOLLOWER
I20260808 02:15:26.586436 8888 consensus_queue.cc:260] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:26.586675 8888 ts_tablet_manager.cc:1428] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:15:26.586691 8678 heartbeater.cc:503] Master 127.19.58.254:39339 was elected leader, sending a full tablet report...
W20260808 02:15:26.667292 8681 tablet.cc:2369] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:15:26.691615 8893 raft_consensus.cc:482] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:26.691781 8893 raft_consensus.cc:504] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:26.692160 8893 leader_election.cc:288] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64238340aabc44258e69378025c835c6 (127.19.58.194:35663), 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709)
I20260808 02:15:26.695941 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.692440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:44689 (local address 127.19.58.193:43709)
0808 02:15:26.692487 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.692489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.692499 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:15:26.692527 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.692528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.692551 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.692618 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:26.692781 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.693579 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.694982 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.695612 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.695671 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.695775 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.695840 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.695881 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:15:26.695940 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.692465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:59481 (local address 127.19.58.194:35663)
0808 02:15:26.692512 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.692514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.692606 (+ 92us) server_negotiation.cc:407] Connection header received
0808 02:15:26.692651 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.692652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.692663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.692708 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:26.692851 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.693570 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.694981 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.695557 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.695655 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.695666 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.695802 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.695846 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:15:26.695942 8877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.692329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:43709 (local address 127.19.58.195:44689)
0808 02:15:26.692386 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.692398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:26.692448 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:26.692621 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:26.692622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:26.692623 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:26.692701 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.692703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.693651 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.693653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.694893 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.694896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.695720 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.695721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.695788 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.695791 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:15:26.695837 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:15:26.695867 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":7}
I20260808 02:15:26.695951 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.692340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35663 (local address 127.19.58.195:59481)
0808 02:15:26.692546 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.692554 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:26.692603 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:26.692722 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:26.692723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:26.692724 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:26.692787 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.692788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.693651 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.693653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.694893 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:26.694896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.695635 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.695635 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:26.695736 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.695740 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:15:26.695795 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 02:15:26.695856 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":103,"thread_start_us":149,"threads_started":1}
I20260808 02:15:26.696319 8421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b88ff0ab734857bb5ac7aeef98cfbd" candidate_uuid: "0b41674c945041c988290ec858fe8394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089b59afeb6649db814db2198ee396d3" is_pre_election: true
I20260808 02:15:26.696339 8620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b88ff0ab734857bb5ac7aeef98cfbd" candidate_uuid: "0b41674c945041c988290ec858fe8394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64238340aabc44258e69378025c835c6" is_pre_election: true
I20260808 02:15:26.696484 8421 raft_consensus.cc:2454] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b41674c945041c988290ec858fe8394 in term 0.
I20260808 02:15:26.696497 8620 raft_consensus.cc:2454] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b41674c945041c988290ec858fe8394 in term 0.
I20260808 02:15:26.696712 8713 leader_election.cc:302] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b41674c945041c988290ec858fe8394, 64238340aabc44258e69378025c835c6; no voters:
I20260808 02:15:26.696869 8893 raft_consensus.cc:2781] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:26.696942 8893 raft_consensus.cc:482] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:26.696959 8893 raft_consensus.cc:3037] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:26.697714 8893 raft_consensus.cc:504] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:26.697890 8893 leader_election.cc:288] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [CANDIDATE]: Term 1 election: Requested vote from peers 64238340aabc44258e69378025c835c6 (127.19.58.194:35663), 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709)
I20260808 02:15:26.697969 8620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b88ff0ab734857bb5ac7aeef98cfbd" candidate_uuid: "0b41674c945041c988290ec858fe8394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64238340aabc44258e69378025c835c6"
I20260808 02:15:26.698025 8421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b88ff0ab734857bb5ac7aeef98cfbd" candidate_uuid: "0b41674c945041c988290ec858fe8394" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089b59afeb6649db814db2198ee396d3"
I20260808 02:15:26.698048 8620 raft_consensus.cc:3037] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:26.698107 8421 raft_consensus.cc:3037] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:26.698756 8421 raft_consensus.cc:2454] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b41674c945041c988290ec858fe8394 in term 1.
I20260808 02:15:26.698760 8620 raft_consensus.cc:2454] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b41674c945041c988290ec858fe8394 in term 1.
I20260808 02:15:26.698916 8708 leader_election.cc:302] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 089b59afeb6649db814db2198ee396d3, 0b41674c945041c988290ec858fe8394; no voters:
I20260808 02:15:26.699028 8893 raft_consensus.cc:2781] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:26.699163 8893 raft_consensus.cc:686] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 1 LEADER]: Becoming Leader. State: Replica: 0b41674c945041c988290ec858fe8394, State: Running, Role: LEADER
I20260808 02:15:26.699244 8893 consensus_queue.cc:237] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:26.700080 8230 catalog_manager.cc:5495] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b41674c945041c988290ec858fe8394 (127.19.58.195). New cstate: current_term: 1 leader_uuid: "0b41674c945041c988290ec858fe8394" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } health_report { overall_health: UNKNOWN } } }
I20260808 02:15:26.721068 8883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.718851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.19.58.195:45021)
0808 02:15:26.718907 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:26.718909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:26.718918 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:15:26.718968 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:26.718969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:26.718988 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:26.719047 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:15:26.719165 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.719634 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.720356 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:26.720548 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:26.720603 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:26.720748 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:15:26.720850 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:15:26.720920 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:26.720962 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 02:15:26.720981 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:15:26.722704 8620 raft_consensus.cc:1261] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Refusing update from remote peer 0b41674c945041c988290ec858fe8394: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:26.722704 8421 raft_consensus.cc:1261] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Refusing update from remote peer 0b41674c945041c988290ec858fe8394: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:26.723057 8896 consensus_queue.cc:1035] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:26.723170 8893 consensus_queue.cc:1035] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:26.725770 8906 mvcc.cc:205] Tried to move back new op lower bound from 7316092218253697024 to 7316092218160766976. Current Snapshot: MvccSnapshot[applied={T|T < 7316092218253697024}]
I20260808 02:15:26.727911 8230 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:26.728009 8230 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:26.728704 8230 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Sending DeleteTablet for 3 replicas of tablet e5b88ff0ab734857bb5ac7aeef98cfbd
I20260808 02:15:26.729048 8396 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b88ff0ab734857bb5ac7aeef98cfbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:26 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:50010
I20260808 02:15:26.729053 8781 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b88ff0ab734857bb5ac7aeef98cfbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:26 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:40074
I20260808 02:15:26.729068 8588 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b88ff0ab734857bb5ac7aeef98cfbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:26 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:52052
I20260808 02:15:26.729149 8497 tablet_replica.cc:330] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: stopping tablet replica
I20260808 02:15:26.729199 8497 raft_consensus.cc:2229] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:26.729254 8497 raft_consensus.cc:2258] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:26.729254 8919 tablet_replica.cc:330] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: stopping tablet replica
I20260808 02:15:26.729279 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 8283
I20260808 02:15:26.729338 8919 raft_consensus.cc:2229] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:15:26.729442 8919 raft_consensus.cc:2258] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:26.736337 8919 ts_tablet_manager.cc:1905] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:26.738307 8919 ts_tablet_manager.cc:1918] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:26.738391 8919 log.cc:1196] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/e5b88ff0ab734857bb5ac7aeef98cfbd
I20260808 02:15:26.738696 8919 ts_tablet_manager.cc:1939] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Deleting consensus metadata
I20260808 02:15:26.739096 8212 catalog_manager.cc:4841] TS 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021): tablet e5b88ff0ab734857bb5ac7aeef98cfbd (table test-table [id=58520b14285344f8a2106e9aa9abb9c6]) successfully deleted
W20260808 02:15:26.746596 8212 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:43709 (error 108)
I20260808 02:15:26.746737 8920 tablet_replica.cc:330] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: stopping tablet replica
I20260808 02:15:26.746853 8920 raft_consensus.cc:2229] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:26.746968 8920 raft_consensus.cc:2258] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:26.746994 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 8490
I20260808 02:15:26.747328 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.747210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:43709 (local address 127.0.0.1:50014)
0808 02:15:26.747257 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.747292 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:43709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 02:15:26.747522 8212 catalog_manager.cc:4584] TS 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5b88ff0ab734857bb5ac7aeef98cfbd: Network error: Client connection negotiation failed: client connection to 127.19.58.193:43709: connect: Connection refused (error 111)
I20260808 02:15:26.753855 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 8684
I20260808 02:15:26.753899 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:26.753790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35663 (local address 127.0.0.1:52056)
0808 02:15:26.753846 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:26.753871 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.194:35663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 02:15:26.765093 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 8188
I20260808 02:15:26.774824 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:39339
--webserver_interface=127.19.58.254
--webserver_port=37517
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:39339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:26.855482 8926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:26.855674 8926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:26.855693 8926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:26.856834 8926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:26.856877 8926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:26.856889 8926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:26.856904 8926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:26.856930 8926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:26.858358 8926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:39339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:39339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=37517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:26.858584 8926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:26.858793 8926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:26.863366 8936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.863477 8932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:26.863379 8933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:26.863598 8926 server_base.cc:1034] running on GCE node
I20260808 02:15:26.863839 8926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:26.864075 8926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:26.865202 8926 hybrid_clock.cc:648] HybridClock initialized: now 1786155326865191 us; error 29 us; skew 500 ppm
Aug 08 02:15:26 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155326, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:26.867575 8926 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:15:26.868022 8926 webserver.cc:460] Webserver started at http://127.19.58.254:37517/ using document root <none> and password file <none>
I20260808 02:15:26.868228 8926 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:26.868273 8926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:26.869520 8926 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:26.870128 8943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.870265 8926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:26.870338 8926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "ef8890302dc8418e8083f1f210033001"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "0d513d5eccddd0594f007d812f6d112e"
server_key_iv: "0c21e2d3f3345a976f3845c5eee47a22"
server_key_version: "encryptionkey@0"
I20260808 02:15:26.870641 8926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:26.894289 8926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:26.896416 8926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:26.896550 8926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:26.900525 8926 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:39339
I20260808 02:15:26.900563 9014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:39339 every 8 connection(s)
I20260808 02:15:26.900906 8926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:26.902395 9016 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:15:26.903021 9016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Bootstrap starting.
I20260808 02:15:26.909994 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 8926
I20260808 02:15:26.910322 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:43709
--local_ip_for_outbound_sockets=127.19.58.193
--tserver_master_addrs=127.19.58.254:39339
--webserver_port=43157
--webserver_interface=127.19.58.193
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:26.912626 9016 log.cc:824] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:26.914803 9016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:26.915076 9016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001: Bootstrap complete.
I20260808 02:15:26.917233 9016 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:26.917459 9016 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8890302dc8418e8083f1f210033001, State: Initialized, Role: FOLLOWER
I20260808 02:15:26.917598 9016 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:26.917661 9016 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:26.917701 9016 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:26.917742 9016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 2 FOLLOWER]: Advancing to term 3
I20260808 02:15:26.918721 9016 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:26.918812 9016 leader_election.cc:302] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef8890302dc8418e8083f1f210033001; no voters:
I20260808 02:15:26.918958 9016 leader_election.cc:288] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 02:15:26.919024 9021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 02:15:26.919162 9021 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [term 3 LEADER]: Becoming Leader. State: Replica: ef8890302dc8418e8083f1f210033001, State: Running, Role: LEADER
I20260808 02:15:26.919209 9016 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:26.919262 9021 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } }
I20260808 02:15:26.919515 9024 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef8890302dc8418e8083f1f210033001. Latest consensus state: current_term: 3 leader_uuid: "ef8890302dc8418e8083f1f210033001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } } }
I20260808 02:15:26.919518 9023 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ef8890302dc8418e8083f1f210033001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8890302dc8418e8083f1f210033001" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 39339 } } }
I20260808 02:15:26.919693 9023 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:26.919680 9024 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8890302dc8418e8083f1f210033001 [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:26.920182 9033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:26.920948 9033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:26.921046 9033 catalog_manager.cc:1259] Loaded cluster ID: 34f3b1b59ede4aff90dc3bc81104700b
I20260808 02:15:26.921070 9033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 02:15:26.921352 9033 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbe7a6b29ad google::LogMessage::Fail() at ??:0
@ 0x7fbe7a6b588a google::LogMessage::SendToLog() at ??:0
@ 0x7fbe7a6b24de google::LogMessage::Flush() at ??:0
@ 0x7fbe7a6b3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbe825dc0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fbe825e58b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbe7c5800d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbe7c575954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbe7d3d96db start_thread at ??:0
@ 0x7fbe791b061f clone at ??:0
W20260808 02:15:26.996834 9019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:26.997017 9019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:26.997049 9019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:26.998135 9019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:26.998174 9019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:26.998208 9019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:15:26.999601 9019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:43709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=43157
--tserver_master_addrs=127.19.58.254:39339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:26.999840 9019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:27.000043 9019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:27.004870 9066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:27.004870 9069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:27.005024 9067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:27.005246 9019 server_base.cc:1034] running on GCE node
I20260808 02:15:27.005427 9019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:27.005674 9019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:27.006816 9019 hybrid_clock.cc:648] HybridClock initialized: now 1786155327006805 us; error 33 us; skew 500 ppm
Aug 08 02:15:27 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:27.009286 9019 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:15:27.009754 9019 webserver.cc:460] Webserver started at http://127.19.58.193:43157/ using document root <none> and password file <none>
I20260808 02:15:27.009981 9019 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:27.010030 9019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:27.011288 9019 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:27.011945 9080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.012130 9019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:15:27.012233 9019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "089b59afeb6649db814db2198ee396d3"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "294a51e4dc3f54df08575cedb55345c2"
server_key_iv: "72170bd952cab6dcee7daa9362c54bf2"
server_key_version: "encryptionkey@0"
I20260808 02:15:27.012681 9019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:27.051118 9019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:27.054011 9019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:27.054173 9019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:27.054448 9019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:27.054953 9090 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:15:27.056454 9019 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:15:27.056516 9019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 02:15:27.056552 9019 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:15:27.057441 9019 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:15:27.057494 9019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 02:15:27.057536 9090 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Bootstrap starting.
I20260808 02:15:27.065111 9019 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:43709
I20260808 02:15:27.065299 9246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:43709 every 8 connection(s)
I20260808 02:15:27.065589 9019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:27.066510 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 9019
I20260808 02:15:27.066828 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:35663
--local_ip_for_outbound_sockets=127.19.58.194
--tserver_master_addrs=127.19.58.254:39339
--webserver_port=46693
--webserver_interface=127.19.58.194
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:27.066977 9090 log.cc:824] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:27.070562 9090 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:27.070873 9090 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Bootstrap complete.
I20260808 02:15:27.072003 9090 ts_tablet_manager.cc:1397] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260808 02:15:27.073356 9090 raft_consensus.cc:348] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:27.073671 9090 raft_consensus.cc:729] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089b59afeb6649db814db2198ee396d3, State: Initialized, Role: FOLLOWER
I20260808 02:15:27.073814 9090 consensus_queue.cc:260] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:27.074075 9090 ts_tablet_manager.cc:1428] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260808 02:15:27.114722 9249 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:15:27.066330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:52113)
0808 02:15:27.067826 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.067871 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:27.068453 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:27.114522 (+ 46069us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1455,"thread_start_us":96,"threads_started":1}
W20260808 02:15:27.114985 9247 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:39339 (0 consecutive failures): Network error: Failed to ping master at 127.19.58.254:39339: Client connection negotiation failed: client connection to 127.19.58.254:39339: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:15:27.115339 9249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.115204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:33857)
0808 02:15:27.115255 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.115312 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 02:15:27.115729 9249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.115614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:53973)
0808 02:15:27.115657 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.115681 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 02:15:27.115851 9247 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:15:27.148238 9254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:27.148423 9254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:27.148444 9254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:27.149580 9254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:27.149621 9254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:27.149645 9254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:15:27.151087 9254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:35663
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=46693
--tserver_master_addrs=127.19.58.254:39339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:27.151300 9254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:27.151508 9254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:27.156548 9261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:27.156558 9262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:27.156611 9265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:27.156772 9254 server_base.cc:1034] running on GCE node
I20260808 02:15:27.156966 9254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:27.157182 9254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:27.158351 9254 hybrid_clock.cc:648] HybridClock initialized: now 1786155327158329 us; error 46 us; skew 500 ppm
Aug 08 02:15:27 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:27.160848 9254 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:15:27.161278 9254 webserver.cc:460] Webserver started at http://127.19.58.194:46693/ using document root <none> and password file <none>
I20260808 02:15:27.161521 9254 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:27.161573 9254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:27.162823 9254 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:27.163511 9274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.163662 9254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:15:27.163764 9254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "64238340aabc44258e69378025c835c6"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "baee51fd7bc78aff4b4e2c5fd6bbf4de"
server_key_iv: "257105b51f61973319b8e34801303836"
server_key_version: "encryptionkey@0"
I20260808 02:15:27.164072 9254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:27.178275 9254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:27.180321 9254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:27.180450 9254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:27.180642 9254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:27.181021 9288 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:15:27.182417 9254 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:15:27.182464 9254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 02:15:27.182494 9254 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:15:27.183108 9254 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:15:27.183142 9254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 02:15:27.183223 9288 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Bootstrap starting.
I20260808 02:15:27.190253 9254 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:35663
I20260808 02:15:27.190348 9435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:35663 every 8 connection(s)
I20260808 02:15:27.190655 9254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:27.191802 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.191237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:55447)
0808 02:15:27.191444 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.191547 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
W20260808 02:15:27.192019 9436 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:39339 (0 consecutive failures): Network error: Failed to ping master at 127.19.58.254:39339: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
I20260808 02:15:27.192322 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.192181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:54109)
0808 02:15:27.192248 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.192293 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 02:15:27.192548 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 9254
I20260808 02:15:27.192658 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.192554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:45325)
0808 02:15:27.192605 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.192636 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 02:15:27.192760 9436 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:15:27.192818 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:45021
--local_ip_for_outbound_sockets=127.19.58.195
--tserver_master_addrs=127.19.58.254:39339
--webserver_port=39099
--webserver_interface=127.19.58.195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:15:27.193275 9288 log.cc:824] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:27.195745 9288 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:27.196003 9288 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Bootstrap complete.
I20260808 02:15:27.196831 9288 ts_tablet_manager.cc:1397] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.009s
I20260808 02:15:27.198400 9288 raft_consensus.cc:348] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:27.198761 9288 raft_consensus.cc:729] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64238340aabc44258e69378025c835c6, State: Initialized, Role: FOLLOWER
I20260808 02:15:27.198920 9288 consensus_queue.cc:260] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:27.199187 9288 ts_tablet_manager.cc:1428] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:15:27.199407 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.199292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:44223)
0808 02:15:27.199346 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.199384 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 02:15:27.199790 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.199673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:60429)
0808 02:15:27.199736 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.199765 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 02:15:27.276755 9444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:27.276938 9444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:27.276960 9444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:27.278010 9444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:27.278043 9444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:27.278066 9444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:15:27.279488 9444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:45021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=39099
--tserver_master_addrs=127.19.58.254:39339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:27.279667 9444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:27.279982 9444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:27.284610 9455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:27.284616 9450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:27.284686 9453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:27.284848 9444 server_base.cc:1034] running on GCE node
I20260808 02:15:27.285049 9444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:27.285254 9444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:27.286394 9444 hybrid_clock.cc:648] HybridClock initialized: now 1786155327286383 us; error 33 us; skew 500 ppm
Aug 08 02:15:27 dist-test-slave-1l3l krb5kdc[7418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:27.288854 9444 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:15:27.289235 9444 webserver.cc:460] Webserver started at http://127.19.58.195:39099/ using document root <none> and password file <none>
I20260808 02:15:27.289460 9444 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:27.289506 9444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:27.290760 9444 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:27.291292 9468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.291460 9444 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:15:27.291539 9444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "0b41674c945041c988290ec858fe8394"
format_stamp: "Formatted at 2026-08-08 02:15:25 on dist-test-slave-1l3l"
server_key: "8f55e5800f519cb7a2ffb21e615c00bd"
server_key_iv: "db8a4e813db8eac86bcb8c2b481d367e"
server_key_version: "encryptionkey@0"
I20260808 02:15:27.291879 9444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:27.327091 9444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:27.328852 9444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:27.328989 9444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:27.329207 9444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:27.329494 9444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:27.329528 9444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.329556 9444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:27.329581 9444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:27.334964 9444 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:45021
I20260808 02:15:27.335019 9626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:45021 every 8 connection(s)
I20260808 02:15:27.335338 9444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:15:27.336323 9629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.335965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:52675)
0808 02:15:27.336108 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.336187 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
W20260808 02:15:27.336685 9627 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:39339 (0 consecutive failures): Network error: Failed to ping master at 127.19.58.254:39339: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
I20260808 02:15:27.336941 9629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.336851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:42595)
0808 02:15:27.336899 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.336923 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 02:15:27.337185 9629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:27.337116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:39015)
0808 02:15:27.337150 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:27.337171 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 02:15:27.337271 9627 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:15:27.340380 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 9444
I20260808 02:15:28.116376 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.116150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:42353)
0808 02:15:28.116290 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.116324 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260808 02:15:28.200307 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.200114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:58777)
0808 02:15:28.200232 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.200267 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260808 02:15:28.337780 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.337559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:46323)
0808 02:15:28.337690 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.337725 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260808 02:15:28.803421 9636 raft_consensus.cc:482] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:28.803560 9636 raft_consensus.cc:504] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:28.803922 9636 leader_election.cc:288] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 64238340aabc44258e69378025c835c6 (127.19.58.194:35663), 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021)
Aug 08 02:15:28 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.194@KRBTEST.COM
Aug 08 02:15:28 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.195@KRBTEST.COM
I20260808 02:15:28.815125 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.804170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:51687 (local address 127.19.58.194:35663)
0808 02:15:28.804311 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:28.804324 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:28.804348 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:15:28.804482 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:28.804483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:28.804524 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:28.804622 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:28.805493 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.806868 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.808435 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.809048 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.809163 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.811825 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:28.811830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:28.811830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:28.811840 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:28.813472 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:28.813968 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:28.813971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:28.813971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:28.814083 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:28.814318 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:28.814319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:28.814319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:28.814551 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:28.814621 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:28.814884 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 02:15:28.815003 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":48,"threads_started":1}
I20260808 02:15:28.815124 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.804102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:45021 (local address 127.19.58.193:50167)
0808 02:15:28.804211 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.804220 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:28.804275 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:28.805095 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:28.805096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:28.805239 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:28.805360 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.805365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.806993 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.806994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.808335 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.808337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.809141 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.809141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.809308 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.809897 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:28.809902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:28.812025 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:28.813716 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:28.813716 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:28.813717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:28.813937 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:28.814159 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:28.814160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:28.814161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:28.814260 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:28.814616 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:28.814623 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:28.814738 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 02:15:28.814824 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:15:28.815014 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":63,"thread_start_us":52,"threads_started":1}
I20260808 02:15:28.815122 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.804201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:50167 (local address 127.19.58.195:45021)
0808 02:15:28.804361 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:28.804369 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:28.804402 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:15:28.804910 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:28.804914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:28.804964 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:28.805100 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:28.805479 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.806871 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.808422 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.809055 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.809159 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.812124 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:28.812127 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:28.812127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:28.812137 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:28.813643 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:28.814013 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:28.814015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:28.814015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:28.814098 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:28.814320 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:28.814321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:28.814321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:28.814550 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:28.814621 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:28.814892 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 02:15:28.815011 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260808 02:15:28.815202 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.804053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35663 (local address 127.19.58.193:51687)
0808 02:15:28.804208 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.804220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:28.804277 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:28.804697 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:28.804700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:28.805133 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:28.805359 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.805365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.806991 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.806992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.808334 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:28.808337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:28.809139 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:28.809140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:28.809308 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:28.809856 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:28.809867 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:28.811694 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:28.813555 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:28.813562 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:28.813564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:28.813891 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:28.814145 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:28.814146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:28.814146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:28.814259 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:28.814618 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:28.814625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:28.814737 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 02:15:28.814824 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 02:15:28.815086 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":189,"thread_start_us":82,"threads_started":1}
W20260808 02:15:28.815893 9082 leader_election.cc:341] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021): Not found: Tablet not found: e5b88ff0ab734857bb5ac7aeef98cfbd
I20260808 02:15:28.816118 9376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b88ff0ab734857bb5ac7aeef98cfbd" candidate_uuid: "089b59afeb6649db814db2198ee396d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "64238340aabc44258e69378025c835c6" is_pre_election: true
I20260808 02:15:28.816246 9376 raft_consensus.cc:2454] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 089b59afeb6649db814db2198ee396d3 in term 1.
I20260808 02:15:28.816495 9085 leader_election.cc:302] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 089b59afeb6649db814db2198ee396d3, 64238340aabc44258e69378025c835c6; no voters: 0b41674c945041c988290ec858fe8394
I20260808 02:15:28.816614 9636 raft_consensus.cc:2781] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 02:15:28.816689 9636 raft_consensus.cc:482] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:28.816705 9636 raft_consensus.cc:3037] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:15:28.817874 9636 raft_consensus.cc:504] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:28.818007 9636 leader_election.cc:288] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [CANDIDATE]: Term 2 election: Requested vote from peers 64238340aabc44258e69378025c835c6 (127.19.58.194:35663), 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021)
I20260808 02:15:28.818177 9376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b88ff0ab734857bb5ac7aeef98cfbd" candidate_uuid: "089b59afeb6649db814db2198ee396d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "64238340aabc44258e69378025c835c6"
I20260808 02:15:28.818246 9376 raft_consensus.cc:3037] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 1 FOLLOWER]: Advancing to term 2
W20260808 02:15:28.818326 9082 leader_election.cc:341] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021): Not found: Tablet not found: e5b88ff0ab734857bb5ac7aeef98cfbd
I20260808 02:15:28.819283 9376 raft_consensus.cc:2454] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 089b59afeb6649db814db2198ee396d3 in term 2.
I20260808 02:15:28.819439 9085 leader_election.cc:302] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 089b59afeb6649db814db2198ee396d3, 64238340aabc44258e69378025c835c6; no voters: 0b41674c945041c988290ec858fe8394
I20260808 02:15:28.819523 9636 raft_consensus.cc:2781] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:15:28.819687 9636 raft_consensus.cc:686] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [term 2 LEADER]: Becoming Leader. State: Replica: 089b59afeb6649db814db2198ee396d3, State: Running, Role: LEADER
I20260808 02:15:28.819819 9636 consensus_queue.cc:237] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:28.819969 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:28.819876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:42387)
0808 02:15:28.819915 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:28.819943 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 02:15:29.200982 9644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:29.200744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:59809)
0808 02:15:29.200900 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:29.200932 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260808 02:15:29.338344 9645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:29.338162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:55473)
0808 02:15:29.338274 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:29.338307 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260808 02:15:29.408317 9376 raft_consensus.cc:1261] T e5b88ff0ab734857bb5ac7aeef98cfbd P 64238340aabc44258e69378025c835c6 [term 2 FOLLOWER]: Refusing update from remote peer 089b59afeb6649db814db2198ee396d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260808 02:15:29.408681 9646 consensus_queue.cc:1035] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260808 02:15:29.409575 9644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:29.409473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:44035)
0808 02:15:29.409511 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:29.409540 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 02:15:29.410530 9644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:29.410415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:52607)
0808 02:15:29.410466 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:29.410499 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 02:15:29.410566 9653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:29.410353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:57857)
0808 02:15:29.410483 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:29.410520 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":54,"threads_started":1}
W20260808 02:15:29.410804 9082 consensus_peers.cc:487] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3 -> Peer 0b41674c945041c988290ec858fe8394 (127.19.58.195:45021): Couldn't send request to peer 0b41674c945041c988290ec858fe8394. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5b88ff0ab734857bb5ac7aeef98cfbd. This is attempt 1: this message will repeat every 5th retry.
I20260808 02:15:29.985881 9658 ts_tablet_manager.cc:927] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Initiating tablet copy from peer 089b59afeb6649db814db2198ee396d3 (127.19.58.193:43709)
I20260808 02:15:29.986104 9658 tablet_copy_client.cc:316] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: tablet copy: Beginning tablet copy session from remote peer at address 127.19.58.193:43709
Aug 08 02:15:29 dist-test-slave-1l3l krb5kdc[7418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155327, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:15:29.993392 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:29.986469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:48625 (local address 127.19.58.193:43709)
0808 02:15:29.986586 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:29.986592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:29.986603 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:15:29.986685 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:29.986689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:29.986703 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:29.986761 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:29.987237 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:29.987969 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:29.988703 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:29.989003 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:29.989063 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:29.990945 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:29.990954 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:29.990954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:29.990961 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:29.992268 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:29.992619 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:29.992622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:29.992622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:29.992680 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:29.992839 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:29.992840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:29.992840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:29.992985 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:29.993054 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:29.993218 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 02:15:29.993291 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":47,"threads_started":1}
I20260808 02:15:29.993412 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:29.986355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:43709 (local address 127.19.58.195:48625)
0808 02:15:29.986523 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:29.986544 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:29.986619 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:29.986771 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:29.986776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:29.987079 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:29.987149 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:29.987152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:29.988044 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:29.988045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:29.988626 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:29.988628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:29.989069 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:29.989070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:29.989141 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:29.989580 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:29.989584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:29.990865 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:29.992342 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:29.992348 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:29.992350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:29.992545 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:29.992736 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:29.992736 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:29.992737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:29.992789 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:29.993046 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:29.993053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:29.993105 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 02:15:29.993177 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 02:15:29.993290 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
I20260808 02:15:29.993894 9211 tablet_copy_service.cc:140] P 089b59afeb6649db814db2198ee396d3: Received BeginTabletCopySession request for tablet e5b88ff0ab734857bb5ac7aeef98cfbd from peer 0b41674c945041c988290ec858fe8394 ({username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:48625)
I20260808 02:15:29.993953 9211 tablet_copy_service.cc:161] P 089b59afeb6649db814db2198ee396d3: Beginning new tablet copy session on tablet e5b88ff0ab734857bb5ac7aeef98cfbd from peer 0b41674c945041c988290ec858fe8394 at {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:48625: session id = 0b41674c945041c988290ec858fe8394-e5b88ff0ab734857bb5ac7aeef98cfbd
I20260808 02:15:29.994441 9211 tablet_copy_source_session.cc:205] T e5b88ff0ab734857bb5ac7aeef98cfbd P 089b59afeb6649db814db2198ee396d3: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 02:15:29.995344 9658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b88ff0ab734857bb5ac7aeef98cfbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:29.997846 9658 tablet_copy_client.cc:791] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: tablet copy: Starting download of 0 data blocks...
I20260808 02:15:29.997982 9658 tablet_copy_client.cc:655] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: tablet copy: Starting download of 1 WAL segments...
I20260808 02:15:29.998726 9658 tablet_copy_client.cc:531] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 02:15:29.999403 9658 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Bootstrap starting.
I20260808 02:15:30.000710 9658 log.cc:824] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:30.002882 9658 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:15:30.003089 9658 tablet_bootstrap.cc:492] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Bootstrap complete.
I20260808 02:15:30.003208 9658 ts_tablet_manager.cc:1397] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 02:15:30.004156 9658 raft_consensus.cc:348] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:30.004302 9658 raft_consensus.cc:729] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b41674c945041c988290ec858fe8394, State: Initialized, Role: FOLLOWER
I20260808 02:15:30.004416 9658 consensus_queue.cc:260] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64238340aabc44258e69378025c835c6" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35663 } } peers { permanent_uuid: "0b41674c945041c988290ec858fe8394" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 45021 } } peers { permanent_uuid: "089b59afeb6649db814db2198ee396d3" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 43709 } }
I20260808 02:15:30.004665 9658 ts_tablet_manager.cc:1428] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20260808 02:15:30.004913 9211 tablet_copy_service.cc:342] P 089b59afeb6649db814db2198ee396d3: Request end of tablet copy session 0b41674c945041c988290ec858fe8394-e5b88ff0ab734857bb5ac7aeef98cfbd received from {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:48625
I20260808 02:15:30.004936 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:30.004823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:43855)
0808 02:15:30.004870 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:30.004909 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 02:15:30.004962 9211 tablet_copy_service.cc:433] P 089b59afeb6649db814db2198ee396d3: ending tablet copy session 0b41674c945041c988290ec858fe8394-e5b88ff0ab734857bb5ac7aeef98cfbd on tablet e5b88ff0ab734857bb5ac7aeef98cfbd with peer 0b41674c945041c988290ec858fe8394
I20260808 02:15:30.005218 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:30.005124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:58775)
0808 02:15:30.005167 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:30.005195 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 02:15:30.411123 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:30.410928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:33339)
0808 02:15:30.411043 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:30.411076 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260808 02:15:30.411129 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:30.410923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:57559)
0808 02:15:30.411058 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:30.411088 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
I20260808 02:15:30.579440 9559 raft_consensus.cc:1203] T e5b88ff0ab734857bb5ac7aeef98cfbd P 0b41674c945041c988290ec858fe8394 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260808 02:15:30.579489 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:30.579328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:51279)
0808 02:15:30.579422 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:30.579452 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 02:15:30.580031 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:30.579859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:42921)
0808 02:15:30.579958 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:30.579984 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":46,"threads_started":1}
I20260808 02:15:31.411782 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.411539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:43811)
0808 02:15:31.411671 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.411730 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260808 02:15:31.580150 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.579925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:56497)
0808 02:15:31.580068 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.580101 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260808 02:15:31.580614 9679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:31.580433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:43161)
0808 02:15:31.580544 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:31.580577 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260808 02:15:32.412390 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:32.412200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:50211)
0808 02:15:32.412307 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:32.412341 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
I20260808 02:15:32.580861 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:32.580642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:46953)
0808 02:15:32.580778 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:32.580814 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260808 02:15:32.581216 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:32.581131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:48799)
0808 02:15:32.581171 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:32.581196 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 02:15:32.581218 9684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:32.581006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:46799)
0808 02:15:32.581136 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:32.581170 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":56,"threads_started":1}
I20260808 02:15:33.412989 9685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:33.412792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:40661)
0808 02:15:33.412907 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:33.412943 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":64,"threads_started":1}
I20260808 02:15:33.581804 9686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:33.581584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:37647)
0808 02:15:33.581723 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:33.581759 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
I20260808 02:15:33.581804 9688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:33.581605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:47233)
0808 02:15:33.581725 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:33.581756 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260808 02:15:34.413587 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:34.413355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:59001)
0808 02:15:34.413504 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:34.413540 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260808 02:15:34.582417 9693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:34.582215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:60987)
0808 02:15:34.582337 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:34.582367 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260808 02:15:34.582415 9692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:34.582216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:35747)
0808 02:15:34.582335 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:34.582364 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260808 02:15:35.414191 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:35.413976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:45075)
0808 02:15:35.414108 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:35.414141 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260808 02:15:35.583068 9697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:35.582863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:47459)
0808 02:15:35.582998 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:35.583023 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260808 02:15:35.583068 9698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:35.582820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:57511)
0808 02:15:35.582985 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:35.583021 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":106,"threads_started":1}
I20260808 02:15:36.414781 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:36.414563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:55863)
0808 02:15:36.414698 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:36.414733 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260808 02:15:36.583743 9702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:36.583498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:34765)
0808 02:15:36.583643 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:36.583670 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
I20260808 02:15:36.583740 9703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:36.583487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:43719)
0808 02:15:36.583633 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:36.583671 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260808 02:15:37.415390 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.415188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:48491)
0808 02:15:37.415310 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.415343 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260808 02:15:37.584422 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.584193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:49973)
0808 02:15:37.584338 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.584373 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260808 02:15:37.584435 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:37.584214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:48477)
0808 02:15:37.584358 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:37.584392 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260808 02:15:38.416042 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.415830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:56417)
0808 02:15:38.415956 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.415993 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260808 02:15:38.585038 9709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.584824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:35833)
0808 02:15:38.584960 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.584992 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260808 02:15:38.585045 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:38.584865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:50345)
0808 02:15:38.584974 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:38.585004 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":47,"threads_started":1}
I20260808 02:15:39.416661 9712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.416421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:45463)
0808 02:15:39.416580 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.416614 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260808 02:15:39.585693 9714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.585475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:46741)
0808 02:15:39.585609 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.585643 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260808 02:15:39.585700 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:39.585488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:51659)
0808 02:15:39.585621 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:39.585656 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260808 02:15:40.417310 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:40.417072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:55937)
0808 02:15:40.417209 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:40.417252 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260808 02:15:40.586362 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:40.586132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:46267)
0808 02:15:40.586275 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:40.586313 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260808 02:15:40.586362 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:40.586136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:42641)
0808 02:15:40.586278 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:40.586313 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260808 02:15:41.418041 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:41.417818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.194:50143)
0808 02:15:41.417955 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:41.417992 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260808 02:15:41.586988 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:41.586765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.195:43857)
0808 02:15:41.586904 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:41.586941 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260808 02:15:41.587020 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:41.586805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:39339 (local address 127.19.58.193:43537)
0808 02:15:41.586944 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:41.586980 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:39339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260808 02:15:42.350100 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 9019
I20260808 02:15:42.360335 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 9254
I20260808 02:15:42.368263 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 9444
2026-08-08T02:15:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18579 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:15:42 dist-test-slave-1l3l krb5kdc[9728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:15:42 dist-test-slave-1l3l krb5kdc[9728](info): set up 2 sockets
Aug 08 02:15:42 dist-test-slave-1l3l krb5kdc[9728](info): commencing operation
krb5kdc: starting...
W20260808 02:15:43.483201 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:15:43 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:15:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:15:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:15:43.596774 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42405
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:42405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:43.676546 9748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:43.676745 9748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:43.676765 9748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:43.677893 9748 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:15:43.677933 9748 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:15:43.677944 9748 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:15:43.677955 9748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:15:43.677970 9748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:43.677994 9748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:43.678017 9748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:15:43.678037 9748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:15:43.679486 9748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:42405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:43.679725 9748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:43.679947 9748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:43.684734 9761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:43.684782 9748 server_base.cc:1034] running on GCE node
W20260808 02:15:43.684724 9758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:43.684705 9756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:43.685387 9748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:43.685654 9748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:43.686797 9748 hybrid_clock.cc:648] HybridClock initialized: now 1786155343686786 us; error 28 us; skew 500 ppm
Aug 08 02:15:43 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:43.689198 9748 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:15:43.689664 9748 webserver.cc:460] Webserver started at http://127.19.58.254:44575/ using document root <none> and password file <none>
I20260808 02:15:43.689858 9748 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:43.689904 9748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:43.690027 9748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:43.691116 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "0d892d7e948f46c3991994124f464efa"
format_stamp: "Formatted at 2026-08-08 02:15:43 on dist-test-slave-1l3l"
server_key: "428c634db136a75b2e5084a8b0aba5e3"
server_key_iv: "6bacfb62d58016daab39b8419b003d4b"
server_key_version: "encryptionkey@0"
I20260808 02:15:43.691447 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "0d892d7e948f46c3991994124f464efa"
format_stamp: "Formatted at 2026-08-08 02:15:43 on dist-test-slave-1l3l"
server_key: "428c634db136a75b2e5084a8b0aba5e3"
server_key_iv: "6bacfb62d58016daab39b8419b003d4b"
server_key_version: "encryptionkey@0"
I20260808 02:15:43.692780 9748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:15:43.693596 9770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:43.693768 9748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:43.693863 9748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "0d892d7e948f46c3991994124f464efa"
format_stamp: "Formatted at 2026-08-08 02:15:43 on dist-test-slave-1l3l"
server_key: "428c634db136a75b2e5084a8b0aba5e3"
server_key_iv: "6bacfb62d58016daab39b8419b003d4b"
server_key_version: "encryptionkey@0"
I20260808 02:15:43.693934 9748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:43.706459 9748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:43.708328 9748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:43.708477 9748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:43.712579 9748 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:42405
I20260808 02:15:43.712636 9846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:42405 every 8 connection(s)
I20260808 02:15:43.712997 9748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:15:43.713752 9847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:43.715969 9847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Bootstrap starting.
I20260808 02:15:43.716534 9847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:43.716791 9847 log.cc:824] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:43.717473 9847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: No bootstrap required, opened a new log
I20260808 02:15:43.719015 9847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:15:43.719146 9847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:43.719177 9847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d892d7e948f46c3991994124f464efa, State: Initialized, Role: FOLLOWER
I20260808 02:15:43.719298 9847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:15:43.719364 9847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:15:43.719393 9847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:15:43.719441 9847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:43.719975 9847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:15:43.720068 9847 leader_election.cc:302] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d892d7e948f46c3991994124f464efa; no voters:
I20260808 02:15:43.720217 9847 leader_election.cc:288] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:15:43.720257 9851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:43.720408 9851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 1 LEADER]: Becoming Leader. State: Replica: 0d892d7e948f46c3991994124f464efa, State: Running, Role: LEADER
I20260808 02:15:43.720459 9847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:15:43.720498 9851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:15:43.721035 9855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d892d7e948f46c3991994124f464efa. Latest consensus state: current_term: 1 leader_uuid: "0d892d7e948f46c3991994124f464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } } }
I20260808 02:15:43.721117 9855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:43.721160 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 9748
I20260808 02:15:43.721261 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:15:43.721274 9852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d892d7e948f46c3991994124f464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } } }
I20260808 02:15:43.721374 9862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:15:43.721391 9852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: This master's current role is: LEADER
I20260808 02:15:43.721473 19691 external_mini_cluster.cc:1428] Setting key 68a649679b1c8d71047aae829a818fc9
I20260808 02:15:43.722277 9862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:15:43.723958 9862 catalog_manager.cc:1347] Generated new cluster ID: 7bfa009d32c646a188250dcf1e4c26cc
I20260808 02:15:43.724012 9862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:15:43 dist-test-slave-1l3l krb5kdc[9728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:43.730765 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:43.721940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59014 (local address 127.19.58.254:42405)
0808 02:15:43.722128 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:43.722137 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:43.722291 (+ 154us) server_negotiation.cc:407] Connection header received
0808 02:15:43.722449 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:43.722457 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:43.722514 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:43.722690 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:43.723396 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:43.724540 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:43.725271 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:43.725688 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:43.725779 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:43.727932 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:43.727935 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:43.727936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:43.727946 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:43.729420 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:43.729732 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:43.729734 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:43.729735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:43.729821 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:43.730004 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:43.730005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:43.730006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:43.730220 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:43.730339 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:43.730459 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 02:15:43.730587 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
I20260808 02:15:43.739151 9862 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:15:43.739851 9862 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:15:43.744711 9862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Generated new TSK 0
I20260808 02:15:43.744866 9862 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:15:43.770428 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42405
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:43.850175 9881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:43.850376 9881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:43.850407 9881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:43.851504 9881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:43.851542 9881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:43.851575 9881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:15:43.853018 9881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:43.853237 9881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:43.853452 9881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:43.858135 9886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:43.858165 9887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:43.858273 9881 server_base.cc:1034] running on GCE node
W20260808 02:15:43.858426 9889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:43.858608 9881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:43.858830 9881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:43.859974 9881 hybrid_clock.cc:648] HybridClock initialized: now 1786155343859958 us; error 27 us; skew 500 ppm
Aug 08 02:15:43 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:43.862468 9881 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:15:43.862856 9881 webserver.cc:460] Webserver started at http://127.19.58.193:43585/ using document root <none> and password file <none>
I20260808 02:15:43.863062 9881 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:43.863108 9881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:43.863229 9881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:43.864281 9881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "d122b723bcfe46fd92ec072ca39d66b4"
format_stamp: "Formatted at 2026-08-08 02:15:43 on dist-test-slave-1l3l"
server_key: "45f1e14cbd5547fc383d82f61b73789d"
server_key_iv: "a699fa293e6ab4b6f7e7c6b6955bfe87"
server_key_version: "encryptionkey@0"
I20260808 02:15:43.864609 9881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "d122b723bcfe46fd92ec072ca39d66b4"
format_stamp: "Formatted at 2026-08-08 02:15:43 on dist-test-slave-1l3l"
server_key: "45f1e14cbd5547fc383d82f61b73789d"
server_key_iv: "a699fa293e6ab4b6f7e7c6b6955bfe87"
server_key_version: "encryptionkey@0"
I20260808 02:15:43.865841 9881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:15:43.866590 9898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:43.866766 9881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:15:43.866838 9881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "d122b723bcfe46fd92ec072ca39d66b4"
format_stamp: "Formatted at 2026-08-08 02:15:43 on dist-test-slave-1l3l"
server_key: "45f1e14cbd5547fc383d82f61b73789d"
server_key_iv: "a699fa293e6ab4b6f7e7c6b6955bfe87"
server_key_version: "encryptionkey@0"
I20260808 02:15:43.866899 9881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:43.881719 9881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:43.883442 9881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:43.883570 9881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:43.883806 9881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:43.884101 9881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:43.884135 9881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:43.884164 9881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:43.884188 9881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:43.889621 9881 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:37141
I20260808 02:15:43.889678 10046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:37141 every 8 connection(s)
I20260808 02:15:43.889956 9881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:15:43.895289 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 9881
I20260808 02:15:43.895421 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
Aug 08 02:15:43 dist-test-slave-1l3l krb5kdc[9728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155343, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:43.895637 19691 external_mini_cluster.cc:1428] Setting key 6fdbcb66977f6dd61217a8dc315952b7
I20260808 02:15:43.898722 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:43.890596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:35275 (local address 127.19.58.254:42405)
0808 02:15:43.890648 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:43.890649 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:43.890836 (+ 187us) server_negotiation.cc:407] Connection header received
0808 02:15:43.891281 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:43.891282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:43.891294 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:43.891347 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:43.892248 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:43.892774 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:43.893708 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:43.893911 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:43.893957 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:43.896054 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:43.896056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:43.896056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:43.896064 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:43.897370 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:43.897804 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:43.897805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:43.897806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:43.897855 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:43.898115 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:43.898116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:43.898116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:43.898252 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:43.898318 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:43.898540 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 02:15:43.898629 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:15:43.898758 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:43.890570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:35275)
0808 02:15:43.890726 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:43.890766 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:43.891212 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:43.891401 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:43.891404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:43.891986 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:43.892165 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:43.892169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:43.892844 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:43.892845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:43.893647 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:43.893650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:43.893964 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:43.893965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:43.894052 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:43.894451 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:43.894459 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:43.895968 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:43.897446 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:43.897454 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:43.897456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:43.897721 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:43.897909 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:43.897909 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:43.897910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:43.898044 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:43.898312 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:43.898316 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:43.898427 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 02:15:43.898504 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:15:43.898613 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260808 02:15:43.899461 10047 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42405
I20260808 02:15:43.899544 10047 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:43.899729 10047 heartbeater.cc:511] Master 127.19.58.254:42405 requested a full tablet report, sending...
I20260808 02:15:43.900271 9794 ts_manager.cc:194] Registered new tserver with Master: d122b723bcfe46fd92ec072ca39d66b4 (127.19.58.193:37141)
I20260808 02:15:43.901145 9794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:35275
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:15:43.919005 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42405
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:43.997536 10052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:43.997722 10052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:43.997743 10052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:43.998829 10052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:43.998868 10052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:43.998893 10052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:15:44.000276 10052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:44.000454 10052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:44.000620 10052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:44.005257 10062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:44.005280 10059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:44.005339 10052 server_base.cc:1034] running on GCE node
W20260808 02:15:44.005280 10065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:44.005635 10052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:44.005844 10052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:44.006976 10052 hybrid_clock.cc:648] HybridClock initialized: now 1786155344006960 us; error 29 us; skew 500 ppm
Aug 08 02:15:44 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155344, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:44.009354 10052 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:15:44.009764 10052 webserver.cc:460] Webserver started at http://127.19.58.194:33759/ using document root <none> and password file <none>
I20260808 02:15:44.009959 10052 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:44.010006 10052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:44.010128 10052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:44.011087 10052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "92abc2e72186410499a20b7025ae5968"
format_stamp: "Formatted at 2026-08-08 02:15:44 on dist-test-slave-1l3l"
server_key: "bfcf5e97ff421d38ff6dac23dcefaa65"
server_key_iv: "6d6e6d1aadcc023ab99e2e2a13e0b4b5"
server_key_version: "encryptionkey@0"
I20260808 02:15:44.011415 10052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "92abc2e72186410499a20b7025ae5968"
format_stamp: "Formatted at 2026-08-08 02:15:44 on dist-test-slave-1l3l"
server_key: "bfcf5e97ff421d38ff6dac23dcefaa65"
server_key_iv: "6d6e6d1aadcc023ab99e2e2a13e0b4b5"
server_key_version: "encryptionkey@0"
I20260808 02:15:44.012629 10052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:15:44.013268 10073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:44.013434 10052 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:15:44.013513 10052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "92abc2e72186410499a20b7025ae5968"
format_stamp: "Formatted at 2026-08-08 02:15:44 on dist-test-slave-1l3l"
server_key: "bfcf5e97ff421d38ff6dac23dcefaa65"
server_key_iv: "6d6e6d1aadcc023ab99e2e2a13e0b4b5"
server_key_version: "encryptionkey@0"
I20260808 02:15:44.013572 10052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:44.034621 10052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:44.036258 10052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:44.036394 10052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:44.036598 10052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:44.036866 10052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:44.036901 10052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:44.036928 10052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:44.036952 10052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:44.042120 10052 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:44725
I20260808 02:15:44.042182 10235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:44725 every 8 connection(s)
I20260808 02:15:44.042444 10052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:15:44.043624 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 10052
I20260808 02:15:44.043731 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:15:44.043915 19691 external_mini_cluster.cc:1428] Setting key 95e574bdd5683712d5478609f6c5804f
Aug 08 02:15:44 dist-test-slave-1l3l krb5kdc[9728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155344, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:44.051496 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.043086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:54505 (local address 127.19.58.254:42405)
0808 02:15:44.043146 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:44.043148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:44.043322 (+ 174us) server_negotiation.cc:407] Connection header received
0808 02:15:44.043851 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:44.043853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:44.043866 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:44.043918 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:44.044910 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.045419 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.046341 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.046538 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.046594 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.048770 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:44.048772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:44.048773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:44.048780 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:44.050230 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:44.050616 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:44.050618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:44.050618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:44.050681 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:44.050885 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:44.050886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:44.050886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:44.051024 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:44.051086 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:44.051306 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 02:15:44.051402 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:15:44.051548 10239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.043060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:54505)
0808 02:15:44.043206 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.043240 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:44.043693 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:44.044000 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:44.044003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:44.044633 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:44.044811 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.044816 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.045501 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.045502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.046263 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.046265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.046606 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.046607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.046714 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.047154 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:44.047162 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:44.048676 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:44.050319 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:44.050325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:44.050326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:44.050547 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:44.050746 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:44.050746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:44.050747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:44.050835 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:44.051086 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:44.051089 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:44.051191 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 02:15:44.051262 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 02:15:44.051371 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260808 02:15:44.051996 10236 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42405
I20260808 02:15:44.052095 10236 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:44.052275 10236 heartbeater.cc:511] Master 127.19.58.254:42405 requested a full tablet report, sending...
I20260808 02:15:44.052672 9794 ts_manager.cc:194] Registered new tserver with Master: 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725)
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
I20260808 02:15:44.053062 9794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:54505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:15:44.068356 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42405
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:15:44.147022 10242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:15:44.147215 10242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:15:44.147244 10242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:15:44.148365 10242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:15:44.148414 10242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:15:44.148449 10242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:15:44.149817 10242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:15:44.150038 10242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:15:44.150242 10242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:15:44.154826 10248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:44.154836 10250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:15:44.154842 10255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:15:44.155100 10242 server_base.cc:1034] running on GCE node
I20260808 02:15:44.155289 10242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:15:44.155535 10242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:15:44.156672 10242 hybrid_clock.cc:648] HybridClock initialized: now 1786155344156642 us; error 41 us; skew 500 ppm
Aug 08 02:15:44 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155344, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:15:44.158929 10242 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:15:44.159415 10242 webserver.cc:460] Webserver started at http://127.19.58.195:35983/ using document root <none> and password file <none>
I20260808 02:15:44.159614 10242 fs_manager.cc:359] Metadata directory not provided
I20260808 02:15:44.159662 10242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:15:44.159809 10242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:15:44.160795 10242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "a4be58d54faf4991b3faf816f019a95e"
format_stamp: "Formatted at 2026-08-08 02:15:44 on dist-test-slave-1l3l"
server_key: "b07c4c8d869970df8525f75b59887108"
server_key_iv: "0c84b3a9ab57a80f44874b431faffbb4"
server_key_version: "encryptionkey@0"
I20260808 02:15:44.161132 10242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "a4be58d54faf4991b3faf816f019a95e"
format_stamp: "Formatted at 2026-08-08 02:15:44 on dist-test-slave-1l3l"
server_key: "b07c4c8d869970df8525f75b59887108"
server_key_iv: "0c84b3a9ab57a80f44874b431faffbb4"
server_key_version: "encryptionkey@0"
I20260808 02:15:44.162313 10242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:15:44.163065 10262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:44.163228 10242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:15:44.163307 10242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "a4be58d54faf4991b3faf816f019a95e"
format_stamp: "Formatted at 2026-08-08 02:15:44 on dist-test-slave-1l3l"
server_key: "b07c4c8d869970df8525f75b59887108"
server_key_iv: "0c84b3a9ab57a80f44874b431faffbb4"
server_key_version: "encryptionkey@0"
I20260808 02:15:44.163371 10242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:15:44.184171 10242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:15:44.185811 10242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:15:44.185945 10242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:15:44.186148 10242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:15:44.186439 10242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:15:44.186474 10242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:44.186502 10242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:15:44.186519 10242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:15:44.191896 10242 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:46139
I20260808 02:15:44.191962 10409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:46139 every 8 connection(s)
I20260808 02:15:44.192221 10242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:15:44.192915 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 10242
I20260808 02:15:44.193007 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:15:44.193184 19691 external_mini_cluster.cc:1428] Setting key 9a5666a7acb35af5af0fdd7173a25b22
Aug 08 02:15:44 dist-test-slave-1l3l krb5kdc[9728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155344, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:15:44.200480 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.193346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:56883 (local address 127.19.58.254:42405)
0808 02:15:44.193397 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:44.193399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:44.193410 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:15:44.193596 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:44.193597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:44.193607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:44.193648 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:44.194602 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.195092 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.196092 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.196294 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.196350 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.198265 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:44.198267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:44.198267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:44.198273 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:44.199336 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:44.199675 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:44.199676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:44.199676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:44.199766 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:44.199997 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:44.199998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:44.199998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:44.200097 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:44.200145 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:44.200339 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 02:15:44.200418 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:15:44.200592 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.192814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:56883)
0808 02:15:44.192975 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.193015 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:44.193514 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:44.193720 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:44.193724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:44.194332 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:15:44.194519 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.194523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.195160 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.195160 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.196025 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.196027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.196359 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.196359 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.196469 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.196899 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:15:44.196907 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:15:44.198168 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:15:44.199421 (+ 1253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:44.199427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:44.199428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:44.199622 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:44.199822 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:15:44.199823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:15:44.199823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:15:44.199931 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:15:44.200141 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:15:44.200145 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:15:44.200246 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 02:15:44.200319 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 02:15:44.200431 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260808 02:15:44.201100 10410 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42405
I20260808 02:15:44.201191 10410 heartbeater.cc:464] Registering TS with master...
I20260808 02:15:44.201355 10410 heartbeater.cc:511] Master 127.19.58.254:42405 requested a full tablet report, sending...
I20260808 02:15:44.201682 9794 ts_manager.cc:194] Registered new tserver with Master: a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139)
I20260808 02:15:44.202029 9794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:56883
I20260808 02:15:44.204746 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:15:44.210872 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.206021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59020 (local address 127.19.58.254:42405)
0808 02:15:44.206071 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:44.206073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:44.206156 (+ 83us) server_negotiation.cc:407] Connection header received
0808 02:15:44.206200 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:44.206200 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:44.206208 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:44.206250 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:15:44.206700 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.207130 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.207815 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.207986 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.208036 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.208968 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:15:44.208970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:15:44.208970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:15:44.208975 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:15:44.210062 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:44.210328 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:44.210329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:44.210329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:44.210387 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:15:44.210517 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:15:44.210518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:15:44.210518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:15:44.210632 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:15:44.210698 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:44.210766 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 02:15:44.210818 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 02:15:44.212265 9794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:15:44.213722 9794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:15:44.219312 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.215720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:37141 (local address 127.0.0.1:45284)
0808 02:15:44.216049 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.216058 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:44.216105 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:44.216346 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:44.216347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:44.216350 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:44.216391 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.216392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.217676 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.217677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.218403 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.218406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.219030 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.219031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.219106 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.219120 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 02:15:44.219184 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 02:15:44.219234 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":71,"threads_started":1}
I20260808 02:15:44.219321 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.215816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.19.58.193:37141)
0808 02:15:44.216017 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:44.216020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:44.216120 (+ 100us) server_negotiation.cc:407] Connection header received
0808 02:15:44.216188 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:44.216190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:44.216205 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:44.216268 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:44.216471 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.217579 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.218486 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.218957 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.219018 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.219076 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:44.219204 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 02:15:44.219244 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":74,"threads_started":1}
I20260808 02:15:44.219583 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.215499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44725 (local address 127.0.0.1:41920)
0808 02:15:44.215853 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.215893 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:44.215958 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:44.216189 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:44.216191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:44.216193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:44.216309 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.216313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.217339 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.217340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.218436 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.218438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.219363 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.219364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.219425 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.219428 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:15:44.219476 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 02:15:44.219514 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260808 02:15:44.219619 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.215603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41920 (local address 127.19.58.194:44725)
0808 02:15:44.215864 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:44.215880 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:44.216039 (+ 159us) server_negotiation.cc:407] Connection header received
0808 02:15:44.216086 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:44.216089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:44.216106 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:44.216185 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:44.216389 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.217232 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.218519 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.219223 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.219283 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.219331 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:44.219498 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 02:15:44.219542 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":83,"threads_started":1}
I20260808 02:15:44.220104 10425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.215499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:46139 (local address 127.0.0.1:45022)
0808 02:15:44.215853 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.215899 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:44.215958 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:44.216241 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:44.216242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:44.216245 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:44.216296 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.216298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.217659 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.217660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.218988 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.218991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.219857 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.219857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.219946 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.219949 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:15:44.220004 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 02:15:44.220039 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":25,"thread_start_us":66,"threads_started":1}
I20260808 02:15:44.220117 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.215678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.19.58.195:46139)
0808 02:15:44.216076 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:44.216079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:44.216086 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:15:44.216118 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:44.216120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:44.216135 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:44.216364 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:44.216425 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.217558 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.219073 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.219796 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.219856 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.219951 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:44.220016 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:15:44.220048 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":98,"threads_started":1}
I20260808 02:15:44.220301 9963 tablet_service.cc:1467] Processing CreateTablet for tablet 8beae6fb9d014b4a833f13d01cc50ecf (DEFAULT_TABLE table=test-table [id=2c2db39ef60b45a7bd0edb10fa67ddab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:44.220628 9963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8beae6fb9d014b4a833f13d01cc50ecf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:44.223067 10429 tablet_bootstrap.cc:492] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Bootstrap starting.
I20260808 02:15:44.223364 10141 tablet_service.cc:1467] Processing CreateTablet for tablet 8beae6fb9d014b4a833f13d01cc50ecf (DEFAULT_TABLE table=test-table [id=2c2db39ef60b45a7bd0edb10fa67ddab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:44.223675 10141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8beae6fb9d014b4a833f13d01cc50ecf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:44.223678 10429 tablet_bootstrap.cc:654] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:44.224154 10429 log.cc:824] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:44.224757 10429 tablet_bootstrap.cc:492] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: No bootstrap required, opened a new log
I20260808 02:15:44.224809 10429 ts_tablet_manager.cc:1397] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:15:44.225950 10431 tablet_bootstrap.cc:492] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: Bootstrap starting.
I20260808 02:15:44.226120 10429 raft_consensus.cc:348] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:15:44.226289 10429 raft_consensus.cc:374] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:44.226326 10429 raft_consensus.cc:729] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d122b723bcfe46fd92ec072ca39d66b4, State: Initialized, Role: FOLLOWER
I20260808 02:15:44.226468 10429 consensus_queue.cc:260] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:15:44.226604 10431 tablet_bootstrap.cc:654] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:44.226730 10429 ts_tablet_manager.cc:1428] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:15:44.226920 10318 tablet_service.cc:1467] Processing CreateTablet for tablet 8beae6fb9d014b4a833f13d01cc50ecf (DEFAULT_TABLE table=test-table [id=2c2db39ef60b45a7bd0edb10fa67ddab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:15:44.227146 10431 log.cc:824] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:44.227191 10318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8beae6fb9d014b4a833f13d01cc50ecf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:15:44.227744 10431 tablet_bootstrap.cc:492] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: No bootstrap required, opened a new log
I20260808 02:15:44.227790 10047 heartbeater.cc:503] Master 127.19.58.254:42405 was elected leader, sending a full tablet report...
I20260808 02:15:44.227805 10431 ts_tablet_manager.cc:1397] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:15:44.229216 10431 raft_consensus.cc:348] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:15:44.229341 10431 raft_consensus.cc:374] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:44.229380 10431 raft_consensus.cc:729] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92abc2e72186410499a20b7025ae5968, State: Initialized, Role: FOLLOWER
I20260808 02:15:44.229488 10431 consensus_queue.cc:260] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:15:44.229730 10431 ts_tablet_manager.cc:1428] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:15:44.229775 10236 heartbeater.cc:503] Master 127.19.58.254:42405 was elected leader, sending a full tablet report...
I20260808 02:15:44.229918 10435 tablet_bootstrap.cc:492] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: Bootstrap starting.
I20260808 02:15:44.230454 10435 tablet_bootstrap.cc:654] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: Neither blocks nor log segments found. Creating new log.
I20260808 02:15:44.230932 10435 log.cc:824] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:15:44.231587 10435 tablet_bootstrap.cc:492] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: No bootstrap required, opened a new log
I20260808 02:15:44.231639 10435 ts_tablet_manager.cc:1397] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:15:44.232869 10435 raft_consensus.cc:348] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:15:44.232992 10435 raft_consensus.cc:374] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:15:44.233013 10435 raft_consensus.cc:729] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4be58d54faf4991b3faf816f019a95e, State: Initialized, Role: FOLLOWER
I20260808 02:15:44.233114 10435 consensus_queue.cc:260] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:15:44.233349 10435 ts_tablet_manager.cc:1428] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:15:44.233384 10410 heartbeater.cc:503] Master 127.19.58.254:42405 was elected leader, sending a full tablet report...
I20260808 02:15:44.263779 10432 raft_consensus.cc:482] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:15:44.263875 10432 raft_consensus.cc:504] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:15:44.264158 10432 leader_election.cc:288] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139), 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725)
I20260808 02:15:44.266947 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.264368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:46009 (local address 127.19.58.195:46139)
0808 02:15:44.264431 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:44.264433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:44.264442 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:15:44.264512 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:44.264513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:44.264523 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:44.264576 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:44.264713 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.265241 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.266220 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.266643 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.266696 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.266703 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:44.266851 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 02:15:44.266884 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 02:15:44.266950 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.264323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:46139 (local address 127.19.58.193:46009)
0808 02:15:44.264388 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.264399 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:44.264443 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:44.264579 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:44.264580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:44.264581 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:44.264655 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.264657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.265297 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.265298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.266139 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.266142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.266700 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.266701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.266780 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.266784 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:15:44.266843 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:15:44.266882 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 02:15:44.267105 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.264347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44725 (local address 127.19.58.193:57529)
0808 02:15:44.264503 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.264514 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:15:44.264573 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:15:44.264688 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:15:44.264689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:15:44.264690 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:15:44.264759 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.264761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.265390 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.265391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.266190 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:15:44.266193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.266858 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.266859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:15:44.266930 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.266934 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:15:44.266989 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 02:15:44.267025 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260808 02:15:44.267108 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.264381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:57529 (local address 127.19.58.194:44725)
0808 02:15:44.264428 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:44.264430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:44.264580 (+ 150us) server_negotiation.cc:407] Connection header received
0808 02:15:44.264614 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:44.264615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:44.264626 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:44.264681 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:15:44.264816 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.265318 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.266277 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.266770 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.266842 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.266851 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:44.267005 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 02:15:44.267046 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:15:44.267318 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8beae6fb9d014b4a833f13d01cc50ecf" candidate_uuid: "d122b723bcfe46fd92ec072ca39d66b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4be58d54faf4991b3faf816f019a95e" is_pre_election: true
I20260808 02:15:44.267454 10341 raft_consensus.cc:2454] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d122b723bcfe46fd92ec072ca39d66b4 in term 0.
I20260808 02:15:44.267442 10167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8beae6fb9d014b4a833f13d01cc50ecf" candidate_uuid: "d122b723bcfe46fd92ec072ca39d66b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92abc2e72186410499a20b7025ae5968" is_pre_election: true
I20260808 02:15:44.267572 10167 raft_consensus.cc:2454] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d122b723bcfe46fd92ec072ca39d66b4 in term 0.
I20260808 02:15:44.267653 9900 leader_election.cc:302] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4be58d54faf4991b3faf816f019a95e, d122b723bcfe46fd92ec072ca39d66b4; no voters:
I20260808 02:15:44.267794 10432 raft_consensus.cc:2781] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:15:44.267845 10432 raft_consensus.cc:482] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:15:44.267860 10432 raft_consensus.cc:3037] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:44.268594 10432 raft_consensus.cc:504] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:15:44.268707 10432 leader_election.cc:288] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 1 election: Requested vote from peers a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139), 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725)
I20260808 02:15:44.268887 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8beae6fb9d014b4a833f13d01cc50ecf" candidate_uuid: "d122b723bcfe46fd92ec072ca39d66b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4be58d54faf4991b3faf816f019a95e"
I20260808 02:15:44.268915 10167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8beae6fb9d014b4a833f13d01cc50ecf" candidate_uuid: "d122b723bcfe46fd92ec072ca39d66b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92abc2e72186410499a20b7025ae5968"
I20260808 02:15:44.268981 10341 raft_consensus.cc:3037] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:44.268981 10167 raft_consensus.cc:3037] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:15:44.269834 10341 raft_consensus.cc:2454] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d122b723bcfe46fd92ec072ca39d66b4 in term 1.
I20260808 02:15:44.269834 10167 raft_consensus.cc:2454] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d122b723bcfe46fd92ec072ca39d66b4 in term 1.
I20260808 02:15:44.269991 9900 leader_election.cc:302] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4be58d54faf4991b3faf816f019a95e, d122b723bcfe46fd92ec072ca39d66b4; no voters:
I20260808 02:15:44.270080 10432 raft_consensus.cc:2781] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:15:44.270205 10432 raft_consensus.cc:686] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 1 LEADER]: Becoming Leader. State: Replica: d122b723bcfe46fd92ec072ca39d66b4, State: Running, Role: LEADER
I20260808 02:15:44.270277 10432 consensus_queue.cc:237] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:15:44.271051 9794 catalog_manager.cc:5495] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d122b723bcfe46fd92ec072ca39d66b4 (127.19.58.193). New cstate: current_term: 1 leader_uuid: "d122b723bcfe46fd92ec072ca39d66b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } health_report { overall_health: UNKNOWN } } }
I20260808 02:15:44.275990 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.273782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45286 (local address 127.19.58.193:37141)
0808 02:15:44.273825 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 02:15:44.273826 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:15:44.273835 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:15:44.273904 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:15:44.273905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:15:44.273921 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:15:44.273981 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:15:44.274092 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.274523 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.275240 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:15:44.275450 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:15:44.275503 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:15:44.275658 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:15:44.275790 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:15:44.275867 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:15:44.275892 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 02:15:44.275910 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:15:44.277570 10167 raft_consensus.cc:1261] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968 [term 1 FOLLOWER]: Refusing update from remote peer d122b723bcfe46fd92ec072ca39d66b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:44.277807 10341 raft_consensus.cc:1261] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e [term 1 FOLLOWER]: Refusing update from remote peer d122b723bcfe46fd92ec072ca39d66b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:15:44.277865 10432 consensus_queue.cc:1035] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:44.278117 10432 consensus_queue.cc:1035] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:15:44.280185 10445 mvcc.cc:205] Tried to move back new op lower bound from 7316092290158583808 to 7316092290131582976. Current Snapshot: MvccSnapshot[applied={T|T < 7316092290158583808}]
I20260808 02:15:44.280869 10446 mvcc.cc:205] Tried to move back new op lower bound from 7316092290158583808 to 7316092290131582976. Current Snapshot: MvccSnapshot[applied={T|T < 7316092290158583808 or (T in {7316092290158583808})}]
I20260808 02:15:44.281023 10447 mvcc.cc:205] Tried to move back new op lower bound from 7316092290158583808 to 7316092290131582976. Current Snapshot: MvccSnapshot[applied={T|T < 7316092290158583808}]
I20260808 02:15:44.282418 9794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:44.282495 9794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:15:44.283138 9794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Sending DeleteTablet for 3 replicas of tablet 8beae6fb9d014b4a833f13d01cc50ecf
I20260808 02:15:44.283456 10318 tablet_service.cc:1514] Processing DeleteTablet for tablet 8beae6fb9d014b4a833f13d01cc50ecf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:44 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:45022
I20260808 02:15:44.283461 9963 tablet_service.cc:1514] Processing DeleteTablet for tablet 8beae6fb9d014b4a833f13d01cc50ecf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:44 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:45284
I20260808 02:15:44.283469 10141 tablet_service.cc:1514] Processing DeleteTablet for tablet 8beae6fb9d014b4a833f13d01cc50ecf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:44 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:41920
I20260808 02:15:44.283699 10455 tablet_replica.cc:330] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: stopping tablet replica
I20260808 02:15:44.283762 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 9881
I20260808 02:15:44.283782 10456 tablet_replica.cc:330] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: stopping tablet replica
I20260808 02:15:44.283864 10456 raft_consensus.cc:2229] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:15:44.283951 10456 raft_consensus.cc:2258] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:44.289402 10455 raft_consensus.cc:2229] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:44.289490 10455 raft_consensus.cc:2258] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:44.290159 10455 ts_tablet_manager.cc:1905] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:44.291527 10455 ts_tablet_manager.cc:1918] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:44.291574 10455 log.cc:1196] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/8beae6fb9d014b4a833f13d01cc50ecf
I20260808 02:15:44.291816 10455 ts_tablet_manager.cc:1939] T 8beae6fb9d014b4a833f13d01cc50ecf P a4be58d54faf4991b3faf816f019a95e: Deleting consensus metadata
I20260808 02:15:44.297248 9774 catalog_manager.cc:4841] TS a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139): tablet 8beae6fb9d014b4a833f13d01cc50ecf (table test-table [id=2c2db39ef60b45a7bd0edb10fa67ddab]) successfully deleted
I20260808 02:15:44.300685 10458 tablet_replica.cc:330] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: stopping tablet replica
I20260808 02:15:44.300768 10458 raft_consensus.cc:2229] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:15:44.300838 10458 raft_consensus.cc:2258] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:15:44.301591 10458 ts_tablet_manager.cc:1905] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:15:44.303021 10458 ts_tablet_manager.cc:1918] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:15:44.303083 10458 log.cc:1196] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/8beae6fb9d014b4a833f13d01cc50ecf
I20260808 02:15:44.303300 10458 ts_tablet_manager.cc:1939] T 8beae6fb9d014b4a833f13d01cc50ecf P 92abc2e72186410499a20b7025ae5968: Deleting consensus metadata
I20260808 02:15:44.303730 9776 catalog_manager.cc:4841] TS 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725): tablet 8beae6fb9d014b4a833f13d01cc50ecf (table test-table [id=2c2db39ef60b45a7bd0edb10fa67ddab]) successfully deleted
W20260808 02:15:44.306883 9774 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:37141 (error 108)
I20260808 02:15:44.307160 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 10052
I20260808 02:15:44.308496 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:15:44.307231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:37141 (local address 127.0.0.1:45300)
0808 02:15:44.308420 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0808 02:15:44.308465 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:37141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1166}
W20260808 02:15:44.312520 9774 catalog_manager.cc:4584] TS d122b723bcfe46fd92ec072ca39d66b4 (127.19.58.193:37141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8beae6fb9d014b4a833f13d01cc50ecf: Network error: Client connection negotiation failed: client connection to 127.19.58.193:37141: connect: Connection refused (error 111)
I20260808 02:15:44.319178 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 10242
I20260808 02:15:44.334273 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 9748
I20260808 02:16:04.342509 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42405
--webserver_interface=127.19.58.254
--webserver_port=44575
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:42405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:04.423204 10460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:04.423388 10460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:04.423409 10460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:04.424546 10460 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:16:04.424584 10460 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:16:04.424597 10460 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:16:04.424607 10460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:04.424621 10460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:04.424646 10460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:04.424666 10460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:04.424686 10460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:04.426127 10460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:42405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=44575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:04.426349 10460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:04.426549 10460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:04.435113 10471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:04.435107 10474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:04.435156 10460 server_base.cc:1034] running on GCE node
W20260808 02:16:04.435107 10478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:04.435586 10460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:04.435909 10460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:04.437062 10460 hybrid_clock.cc:648] HybridClock initialized: now 1786155364437047 us; error 30 us; skew 500 ppm
Aug 08 02:16:04 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155364, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:04.443009 10460 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:16:04.443421 10460 webserver.cc:460] Webserver started at http://127.19.58.254:44575/ using document root <none> and password file <none>
I20260808 02:16:04.443635 10460 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:04.443699 10460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:04.445089 10460 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:04.445812 10486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:04.445994 10460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:04.446085 10460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "0d892d7e948f46c3991994124f464efa"
format_stamp: "Formatted at 2026-08-08 02:15:43 on dist-test-slave-1l3l"
server_key: "428c634db136a75b2e5084a8b0aba5e3"
server_key_iv: "6bacfb62d58016daab39b8419b003d4b"
server_key_version: "encryptionkey@0"
I20260808 02:16:04.446497 10460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:04.466271 10460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:04.468336 10460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:04.468475 10460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:04.472347 10460 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:42405
I20260808 02:16:04.472388 10564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:42405 every 8 connection(s)
I20260808 02:16:04.472743 10460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:04.474169 10565 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:16:04.474807 10565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Bootstrap starting.
I20260808 02:16:04.478081 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 10460
I20260808 02:16:04.478338 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:37141
--local_ip_for_outbound_sockets=127.19.58.193
--tserver_master_addrs=127.19.58.254:42405
--webserver_port=43585
--webserver_interface=127.19.58.193
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:16:04.484037 10565 log.cc:824] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:04.485831 10565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:16:04.486117 10565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Bootstrap complete.
I20260808 02:16:04.488265 10565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:16:04.488488 10565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d892d7e948f46c3991994124f464efa, State: Initialized, Role: FOLLOWER
I20260808 02:16:04.488627 10565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:16:04.488682 10565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:04.488724 10565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:04.488770 10565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 1 FOLLOWER]: Advancing to term 2
I20260808 02:16:04.489720 10565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:16:04.489805 10565 leader_election.cc:302] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d892d7e948f46c3991994124f464efa; no voters:
I20260808 02:16:04.489967 10565 leader_election.cc:288] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 02:16:04.490015 10570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 2 FOLLOWER]: Leader election won for term 2
I20260808 02:16:04.490180 10570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 2 LEADER]: Becoming Leader. State: Replica: 0d892d7e948f46c3991994124f464efa, State: Running, Role: LEADER
I20260808 02:16:04.490229 10565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:04.490267 10570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:16:04.490546 10571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0d892d7e948f46c3991994124f464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } } }
I20260808 02:16:04.490577 10570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d892d7e948f46c3991994124f464efa. Latest consensus state: current_term: 2 leader_uuid: "0d892d7e948f46c3991994124f464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } } }
I20260808 02:16:04.490701 10571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:04.490729 10570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:04.490954 10577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:04.491799 10577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:04.491945 10577 catalog_manager.cc:1259] Loaded cluster ID: 7bfa009d32c646a188250dcf1e4c26cc
I20260808 02:16:04.492049 10577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:04.493379 10577 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:04.493559 10577 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Loaded TSK: 0
I20260808 02:16:04.503472 10577 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Generated new TSK 1
I20260808 02:16:04.503568 10577 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:16:04.560006 10567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:04.560209 10567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:04.560240 10567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:04.561321 10567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:04.561360 10567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:04.561401 10567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:16:04.562817 10567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:37141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=43585
--tserver_master_addrs=127.19.58.254:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:04.563045 10567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:04.563261 10567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:04.568183 10598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:04.568195 10597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:04.568194 10600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:04.568624 10567 server_base.cc:1034] running on GCE node
I20260808 02:16:04.568804 10567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:04.569027 10567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:04.570163 10567 hybrid_clock.cc:648] HybridClock initialized: now 1786155364570145 us; error 39 us; skew 500 ppm
Aug 08 02:16:04 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155364, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:04.572731 10567 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:16:04.573129 10567 webserver.cc:460] Webserver started at http://127.19.58.193:43585/ using document root <none> and password file <none>
I20260808 02:16:04.573319 10567 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:04.573380 10567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:04.574937 10567 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:04.575686 10607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:04.575924 10567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:04.576009 10567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "d122b723bcfe46fd92ec072ca39d66b4"
format_stamp: "Formatted at 2026-08-08 02:15:43 on dist-test-slave-1l3l"
server_key: "45f1e14cbd5547fc383d82f61b73789d"
server_key_iv: "a699fa293e6ab4b6f7e7c6b6955bfe87"
server_key_version: "encryptionkey@0"
I20260808 02:16:04.576308 10567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:04.600426 10567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:04.602608 10567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:04.602756 10567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:04.603029 10567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:04.603533 10618 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:16:04.605592 10567 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:16:04.605645 10567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 02:16:04.605680 10567 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:16:04.606383 10567 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:16:04.606424 10567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 02:16:04.606501 10618 tablet_bootstrap.cc:492] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Bootstrap starting.
I20260808 02:16:04.613095 10567 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:37141
I20260808 02:16:04.613142 10763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:37141 every 8 connection(s)
I20260808 02:16:04.613513 10567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:04.614118 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 10567
I20260808 02:16:04.614439 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:44725
--local_ip_for_outbound_sockets=127.19.58.194
--tserver_master_addrs=127.19.58.254:42405
--webserver_port=33759
--webserver_interface=127.19.58.194
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:16:04.617348 10618 log.cc:824] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:04.620352 10618 tablet_bootstrap.cc:492] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:16:04.620693 10618 tablet_bootstrap.cc:492] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Bootstrap complete.
I20260808 02:16:04.621806 10618 ts_tablet_manager.cc:1397] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260808 02:16:04.623152 10618 raft_consensus.cc:348] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:16:04.623479 10618 raft_consensus.cc:729] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d122b723bcfe46fd92ec072ca39d66b4, State: Initialized, Role: FOLLOWER
I20260808 02:16:04.623661 10618 consensus_queue.cc:260] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } } peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } }
I20260808 02:16:04.623963 10618 ts_tablet_manager.cc:1428] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
Aug 08 02:16:04 dist-test-slave-1l3l krb5kdc[9728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155364, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:04.628022 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.614222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:50107 (local address 127.19.58.254:42405)
0808 02:16:04.617573 (+ 3351us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:04.617584 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:04.617979 (+ 395us) server_negotiation.cc:407] Connection header received
0808 02:16:04.619497 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:04.619502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:04.619544 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:04.619659 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:04.620295 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.621289 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.622332 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.622702 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.622799 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.624949 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:04.624954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:04.624954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:04.624966 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:04.626438 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:04.626819 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:04.626821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:04.626822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:04.626913 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:04.627139 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:04.627140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:04.627141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:04.627342 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:04.627411 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:04.627699 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 02:16:04.627856 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3291,"thread_start_us":81,"threads_started":1}
I20260808 02:16:04.628022 10769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.614158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:50107)
0808 02:16:04.617671 (+ 3513us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:04.617716 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:04.619300 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:04.619695 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:04.619698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:04.619992 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:04.620180 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.620186 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.621422 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.621423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.622253 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.622256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.622797 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.622798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.622910 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.623399 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:04.623407 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:04.624852 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:04.626517 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:04.626524 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:04.626526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:04.626736 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:04.626971 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:04.626971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:04.626972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:04.627074 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:04.627411 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:04.627418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:04.627542 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 02:16:04.627620 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 02:16:04.627808 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3473,"thread_start_us":98,"threads_started":1}
I20260808 02:16:04.628750 10765 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42405
I20260808 02:16:04.628847 10765 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:04.629015 10765 heartbeater.cc:511] Master 127.19.58.254:42405 requested a full tablet report, sending...
I20260808 02:16:04.629662 10511 ts_manager.cc:194] Registered new tserver with Master: d122b723bcfe46fd92ec072ca39d66b4 (127.19.58.193:37141)
I20260808 02:16:04.630716 10511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:50107
I20260808 02:16:04.631593 10765 heartbeater.cc:503] Master 127.19.58.254:42405 was elected leader, sending a full tablet report...
Aug 08 02:16:04 dist-test-slave-1l3l krb5kdc[9728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155364, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.193@KRBTEST.COM
I20260808 02:16:04.638774 10776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.630294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33614 (local address 127.19.58.193:37141)
0808 02:16:04.630451 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:04.630455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:04.630496 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:16:04.630598 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:04.630600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:04.630613 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:04.630687 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:04.631938 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.632638 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.633681 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.633965 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.634011 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.636072 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:04.636079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:04.636079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:04.636085 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:04.637453 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:04.637941 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:04.637944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:04.637944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:04.638003 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:04.638208 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:04.638208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:04.638209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:04.638337 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:04.638451 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:04.638623 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 02:16:04.638697 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260808 02:16:04.638790 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.630236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:37141 (local address 127.0.0.1:33614)
0808 02:16:04.630392 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:04.630439 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:04.630520 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:04.630696 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:04.630699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:04.631803 (+ 1104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:04.631862 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.631865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.632709 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.632710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.633595 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.633598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.634031 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.634032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.634112 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.634504 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:04.634509 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:04.635996 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:04.637526 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:04.637533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:04.637535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:04.637772 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:04.638081 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:04.638082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:04.638082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:04.638146 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:04.638446 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:04.638452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:04.638501 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 02:16:04.638587 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:16:04.638695 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260808 02:16:04.639323 10673 tablet_service.cc:1514] Processing DeleteTablet for tablet 8beae6fb9d014b4a833f13d01cc50ecf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:15:44 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:33614
I20260808 02:16:04.639864 10777 tablet_replica.cc:330] stopping tablet replica
I20260808 02:16:04.639977 10777 raft_consensus.cc:2229] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:04.640048 10777 raft_consensus.cc:2258] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:04.640337 10777 ts_tablet_manager.cc:1905] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:04.642580 10777 ts_tablet_manager.cc:1918] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:04.642653 10777 log.cc:1196] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal/wals/8beae6fb9d014b4a833f13d01cc50ecf
I20260808 02:16:04.642881 10777 ts_tablet_manager.cc:1939] T 8beae6fb9d014b4a833f13d01cc50ecf P d122b723bcfe46fd92ec072ca39d66b4: Deleting consensus metadata
I20260808 02:16:04.643443 10489 catalog_manager.cc:4841] TS d122b723bcfe46fd92ec072ca39d66b4 (127.19.58.193:37141): tablet 8beae6fb9d014b4a833f13d01cc50ecf (table test-table [id=2c2db39ef60b45a7bd0edb10fa67ddab]) successfully deleted
W20260808 02:16:04.701622 10771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:04.701805 10771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:04.701828 10771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:04.702936 10771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:04.702975 10771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:04.703001 10771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:16:04.704435 10771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:44725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=33759
--tserver_master_addrs=127.19.58.254:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:04.704639 10771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:04.704836 10771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:04.709683 10787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:04.709719 10783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:04.709733 10782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:04.709872 10771 server_base.cc:1034] running on GCE node
I20260808 02:16:04.710088 10771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:04.710291 10771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:04.711437 10771 hybrid_clock.cc:648] HybridClock initialized: now 1786155364711441 us; error 41 us; skew 500 ppm
Aug 08 02:16:04 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155364, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:04.713867 10771 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:16:04.714255 10771 webserver.cc:460] Webserver started at http://127.19.58.194:33759/ using document root <none> and password file <none>
I20260808 02:16:04.714439 10771 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:04.714504 10771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:04.715801 10771 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:04.716342 10799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:04.716483 10771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:04.716575 10771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "92abc2e72186410499a20b7025ae5968"
format_stamp: "Formatted at 2026-08-08 02:15:44 on dist-test-slave-1l3l"
server_key: "bfcf5e97ff421d38ff6dac23dcefaa65"
server_key_iv: "6d6e6d1aadcc023ab99e2e2a13e0b4b5"
server_key_version: "encryptionkey@0"
I20260808 02:16:04.716864 10771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:04.727461 10771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:04.729174 10771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:04.729288 10771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:04.729478 10771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:04.729769 10771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:04.729802 10771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:04.729822 10771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:04.729835 10771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:04.735175 10771 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:44725
I20260808 02:16:04.735231 10948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:44725 every 8 connection(s)
I20260808 02:16:04.735548 10771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
Aug 08 02:16:04 dist-test-slave-1l3l krb5kdc[9728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155364, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:04.742501 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 10771
I20260808 02:16:04.742782 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:46139
--local_ip_for_outbound_sockets=127.19.58.195
--tserver_master_addrs=127.19.58.254:42405
--webserver_port=35983
--webserver_interface=127.19.58.195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:16:04.743826 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.736214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:45847 (local address 127.19.58.254:42405)
0808 02:16:04.736274 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:04.736277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:04.736488 (+ 211us) server_negotiation.cc:407] Connection header received
0808 02:16:04.736903 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:04.736904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:04.736917 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:04.736966 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:04.737774 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.738284 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.739239 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.739427 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.739469 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.741432 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:04.741433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:04.741434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:04.741442 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:04.742558 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:04.742924 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:04.742925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:04.742925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:04.742981 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:04.743191 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:04.743192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:04.743192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:04.743298 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:04.743356 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:04.743615 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 02:16:04.743733 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 02:16:04.743943 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.736200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:45847)
0808 02:16:04.736353 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:04.736411 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:04.736831 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:04.737015 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:04.737017 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:04.737533 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:04.737708 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.737712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.738358 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.738359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.739177 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.739180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.739483 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.739484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.739577 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.740042 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:04.740049 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:04.741336 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:04.742635 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:04.742641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:04.742643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:04.742840 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:04.743044 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:04.743045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:04.743045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:04.743132 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:04.743357 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:04.743364 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:04.743475 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 02:16:04.743586 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 02:16:04.743748 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260808 02:16:04.744401 10950 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42405
I20260808 02:16:04.744503 10950 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:04.744680 10950 heartbeater.cc:511] Master 127.19.58.254:42405 requested a full tablet report, sending...
I20260808 02:16:04.745043 10511 ts_manager.cc:194] Registered new tserver with Master: 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725)
I20260808 02:16:04.745443 10511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:45847
W20260808 02:16:04.822441 10953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:04.822647 10953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:04.822679 10953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:04.823793 10953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:04.823832 10953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:04.823865 10953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:16:04.825230 10953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:46139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=35983
--tserver_master_addrs=127.19.58.254:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:04.825446 10953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:04.825654 10953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:04.830359 10960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:04.830367 10962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:04.830341 10959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:04.830444 10953 server_base.cc:1034] running on GCE node
I20260808 02:16:04.830735 10953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:04.830953 10953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:04.832087 10953 hybrid_clock.cc:648] HybridClock initialized: now 1786155364832075 us; error 31 us; skew 500 ppm
Aug 08 02:16:04 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155364, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:04.834447 10953 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:16:04.834828 10953 webserver.cc:460] Webserver started at http://127.19.58.195:35983/ using document root <none> and password file <none>
I20260808 02:16:04.835016 10953 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:04.835084 10953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:04.836391 10953 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260808 02:16:04.836923 10975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:04.837061 10953 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:04.837141 10953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "a4be58d54faf4991b3faf816f019a95e"
format_stamp: "Formatted at 2026-08-08 02:15:44 on dist-test-slave-1l3l"
server_key: "b07c4c8d869970df8525f75b59887108"
server_key_iv: "0c84b3a9ab57a80f44874b431faffbb4"
server_key_version: "encryptionkey@0"
I20260808 02:16:04.837419 10953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:04.845638 10953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:04.847198 10953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:04.847323 10953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:04.847515 10953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:04.847808 10953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:04.847846 10953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:04.847872 10953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:04.847896 10953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:04.854712 10953 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:46139
I20260808 02:16:04.854748 11121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:46139 every 8 connection(s)
I20260808 02:16:04.855041 10953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:16:04.859684 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 10953
Aug 08 02:16:04 dist-test-slave-1l3l krb5kdc[9728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155364, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:04.863802 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.855700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:39149 (local address 127.19.58.254:42405)
0808 02:16:04.855783 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:04.855785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:04.855979 (+ 194us) server_negotiation.cc:407] Connection header received
0808 02:16:04.856444 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:04.856446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:04.856458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:04.856508 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:04.857390 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.857870 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.858813 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.859014 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.859070 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.861270 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:04.861271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:04.861272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:04.861278 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:04.862505 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:04.862891 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:04.862892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:04.862893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:04.862946 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:04.863168 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:04.863170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:04.863170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:04.863277 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:04.863337 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:04.863598 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 02:16:04.863679 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:16:04.863925 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.855676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:39149)
0808 02:16:04.855863 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:04.855900 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:04.856339 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:04.856571 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:04.856575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:04.857101 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:04.857274 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.857279 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.857958 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.857959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.858731 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.858733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.859083 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.859084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.859199 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.859652 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:04.859659 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:04.861181 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:04.862579 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:04.862585 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:04.862588 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:04.862823 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:04.863002 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:04.863003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:04.863003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:04.863102 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:04.863335 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:04.863342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:04.863471 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 02:16:04.863557 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:16:04.863725 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":105,"threads_started":1}
I20260808 02:16:04.864382 11122 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42405
I20260808 02:16:04.864471 11122 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:04.864634 11122 heartbeater.cc:511] Master 127.19.58.254:42405 requested a full tablet report, sending...
I20260808 02:16:04.864986 10511 ts_manager.cc:194] Registered new tserver with Master: a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139)
I20260808 02:16:04.865451 10511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:39149
I20260808 02:16:04.866179 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.860081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.19.58.254:42405)
0808 02:16:04.860208 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:04.860209 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:04.860307 (+ 98us) server_negotiation.cc:407] Connection header received
0808 02:16:04.860351 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:04.860352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:04.860365 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:04.860438 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:04.860978 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.861569 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.862278 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.862467 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.862527 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.863651 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:04.863652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:04.863653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:04.863660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:04.865077 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:04.865424 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:04.865424 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:04.865425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:04.865475 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:04.865691 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:04.865692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:04.865692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:04.865800 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:04.865978 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:04.866018 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 02:16:04.866095 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260808 02:16:04.866838 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:16:04.869138 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.867127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35636 (local address 127.19.58.254:42405)
0808 02:16:04.867163 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:04.867165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:04.867246 (+ 81us) server_negotiation.cc:407] Connection header received
0808 02:16:04.867299 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:04.867299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:04.867325 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:04.867375 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:04.867440 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.867905 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.868576 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.868749 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.868792 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.868895 (+ 103us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:04.868971 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:04.869020 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:04.869081 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 02:16:04.869096 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:16:04.869362 10511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:16:04.870792 10511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:16:04.872900 10673 tablet_service.cc:1467] Processing CreateTablet for tablet 30361401d156421c85d6744047976515 (DEFAULT_TABLE table=test-table [id=ef8133f4874f410f9231ecbb9fb001d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:04.873104 10673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30361401d156421c85d6744047976515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:04.874608 10618 tablet_bootstrap.cc:492] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Bootstrap starting.
I20260808 02:16:04.874986 10618 tablet_bootstrap.cc:654] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:04.875854 10618 tablet_bootstrap.cc:492] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: No bootstrap required, opened a new log
I20260808 02:16:04.875905 10618 ts_tablet_manager.cc:1397] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 02:16:04.876049 10618 raft_consensus.cc:348] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:04.876120 10618 raft_consensus.cc:374] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:04.876163 10618 raft_consensus.cc:729] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d122b723bcfe46fd92ec072ca39d66b4, State: Initialized, Role: FOLLOWER
I20260808 02:16:04.876207 10618 consensus_queue.cc:260] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:04.876291 10618 ts_tablet_manager.cc:1428] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:04.876346 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.872496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:46139 (local address 127.0.0.1:35272)
0808 02:16:04.872648 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:04.872657 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:04.872702 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:04.873001 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:04.873002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:04.873004 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:04.873048 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.873049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.874184 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.874184 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.875198 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.875200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.876103 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.876104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.876167 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.876188 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 02:16:04.876244 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 02:16:04.876284 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260808 02:16:04.876477 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.872386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44725 (local address 127.0.0.1:54526)
0808 02:16:04.872430 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:04.872440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:04.872478 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:04.872868 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:04.872869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:04.872871 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:04.872910 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.872911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.873942 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.873943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.875069 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:04.875072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.876240 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.876241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:04.876313 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.876316 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:04.876369 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 02:16:04.876412 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 02:16:04.876495 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.872491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54526 (local address 127.19.58.194:44725)
0808 02:16:04.872676 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:04.872680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:04.872690 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:16:04.872729 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:04.872731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:04.872748 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:04.872858 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:04.872964 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.873875 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.875162 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.876166 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.876229 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.876278 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:04.876383 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 02:16:04.876417 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
I20260808 02:16:04.876609 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:04.872687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.19.58.195:46139)
0808 02:16:04.872837 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:04.872841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:04.872849 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:16:04.872888 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:04.872889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:04.872903 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:04.873116 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:04.873176 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.874102 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.875278 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:04.876003 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:04.876070 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:04.876122 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:04.876503 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 02:16:04.876542 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260808 02:16:04.877094 10865 tablet_service.cc:1467] Processing CreateTablet for tablet 30361401d156421c85d6744047976515 (DEFAULT_TABLE table=test-table [id=ef8133f4874f410f9231ecbb9fb001d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:04.877375 10865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30361401d156421c85d6744047976515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:04.877410 11038 tablet_service.cc:1467] Processing CreateTablet for tablet 30361401d156421c85d6744047976515 (DEFAULT_TABLE table=test-table [id=ef8133f4874f410f9231ecbb9fb001d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:04.877691 11038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30361401d156421c85d6744047976515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:04.880002 11137 tablet_bootstrap.cc:492] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Bootstrap starting.
I20260808 02:16:04.880352 11138 tablet_bootstrap.cc:492] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: Bootstrap starting.
I20260808 02:16:04.880662 11137 tablet_bootstrap.cc:654] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:04.880878 11138 tablet_bootstrap.cc:654] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:04.881132 11137 log.cc:824] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:04.881351 11138 log.cc:824] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:04.881789 11137 tablet_bootstrap.cc:492] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: No bootstrap required, opened a new log
I20260808 02:16:04.881843 11137 ts_tablet_manager.cc:1397] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:04.883193 11137 raft_consensus.cc:348] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:04.883322 11137 raft_consensus.cc:374] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:04.883340 11137 raft_consensus.cc:729] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92abc2e72186410499a20b7025ae5968, State: Initialized, Role: FOLLOWER
I20260808 02:16:04.883455 11137 consensus_queue.cc:260] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:04.883684 11137 ts_tablet_manager.cc:1428] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:04.883740 10950 heartbeater.cc:503] Master 127.19.58.254:42405 was elected leader, sending a full tablet report...
I20260808 02:16:04.884235 11138 tablet_bootstrap.cc:492] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: No bootstrap required, opened a new log
I20260808 02:16:04.884305 11138 ts_tablet_manager.cc:1397] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 02:16:04.885659 11138 raft_consensus.cc:348] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:04.885783 11138 raft_consensus.cc:374] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:04.885803 11138 raft_consensus.cc:729] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4be58d54faf4991b3faf816f019a95e, State: Initialized, Role: FOLLOWER
I20260808 02:16:04.885885 11138 consensus_queue.cc:260] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:04.886121 11138 ts_tablet_manager.cc:1428] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:04.886157 11122 heartbeater.cc:503] Master 127.19.58.254:42405 was elected leader, sending a full tablet report...
W20260808 02:16:04.986106 10951 tablet.cc:2369] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:16:05.105643 11123 tablet.cc:2369] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:16:05.119302 10767 tablet.cc:2369] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:05.153267 11141 raft_consensus.cc:482] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:05.153237 10773 raft_consensus.cc:482] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:05.153420 10773 raft_consensus.cc:504] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:05.153398 11141 raft_consensus.cc:504] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:05.153774 11141 leader_election.cc:288] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139), d122b723bcfe46fd92ec072ca39d66b4 (127.19.58.193:37141)
I20260808 02:16:05.153788 10773 leader_election.cc:288] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725), a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139)
I20260808 02:16:05.157759 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.154102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:36587 (local address 127.19.58.195:46139)
0808 02:16:05.154157 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:05.154159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:05.154222 (+ 63us) server_negotiation.cc:407] Connection header received
0808 02:16:05.154345 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:05.154346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:05.154356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:05.154384 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:05.154593 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.155419 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.156811 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.157375 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.157421 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.157431 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:05.157653 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 02:16:05.157687 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":27}
I20260808 02:16:05.157768 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.153969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:46139 (local address 127.19.58.194:60641)
0808 02:16:05.154194 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:05.154202 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:05.154238 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:05.154425 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:05.154426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:05.154428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:05.154508 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:05.154511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.155534 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.155535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:05.156708 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:05.156711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.157530 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.157531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:05.157611 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.157614 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:05.157641 (+ 27us) client_negotiation.cc:240] Negotiation successful
0808 02:16:05.157679 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":3}
I20260808 02:16:05.157847 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.154156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:58037 (local address 127.19.58.194:44725)
0808 02:16:05.154198 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:05.154200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:05.154315 (+ 115us) server_negotiation.cc:407] Connection header received
0808 02:16:05.154425 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:05.154426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:05.154434 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:05.154460 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:05.155085 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.155777 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.157008 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.157512 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.157550 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.157557 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:05.157744 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 02:16:05.157774 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 02:16:05.158077 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.154332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:34943 (local address 127.19.58.193:37141)
0808 02:16:05.154573 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:05.154575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:05.154581 (+ 6us) server_negotiation.cc:407] Connection header received
0808 02:16:05.154596 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:05.154597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:05.154605 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:05.154803 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:05.154832 (+ 29us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.155533 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.156861 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.157441 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.157745 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.157802 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:05.157820 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 02:16:05.157987 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":38,"threads_started":1}
I20260808 02:16:05.158221 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.154073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:37141 (local address 127.19.58.194:34943)
0808 02:16:05.154423 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:05.154431 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:05.154471 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:05.154682 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:05.154683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:05.154684 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:05.154745 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:05.154747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.155616 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.155617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:05.156780 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:05.156783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.157538 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.157539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:05.157609 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.157613 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:05.157642 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 02:16:05.157682 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":83,"threads_started":1}
I20260808 02:16:05.158250 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.154147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:60641 (local address 127.19.58.195:46139)
0808 02:16:05.154281 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:05.154283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:05.154291 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:16:05.154320 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:05.154320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:05.154332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:05.154573 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:05.154607 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.155449 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.156804 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.157441 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.158055 (+ 614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.158065 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:05.158084 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 02:16:05.158118 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260808 02:16:05.158260 11143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.153961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:46139 (local address 127.19.58.193:36587)
0808 02:16:05.154098 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:05.154110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:05.154258 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:05.154460 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:05.154460 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:05.154463 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:05.154517 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:05.154520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.155502 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.155503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:05.156709 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:05.156712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.157490 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.157490 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:05.157551 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.157575 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 02:16:05.157845 (+ 270us) client_negotiation.cc:240] Negotiation successful
0808 02:16:05.158201 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260808 02:16:05.158493 11062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30361401d156421c85d6744047976515" candidate_uuid: "92abc2e72186410499a20b7025ae5968" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4be58d54faf4991b3faf816f019a95e" is_pre_election: true
I20260808 02:16:05.158500 11061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30361401d156421c85d6744047976515" candidate_uuid: "d122b723bcfe46fd92ec072ca39d66b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4be58d54faf4991b3faf816f019a95e" is_pre_election: true
I20260808 02:16:05.158638 11062 raft_consensus.cc:2454] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92abc2e72186410499a20b7025ae5968 in term 0.
I20260808 02:16:05.158722 10701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30361401d156421c85d6744047976515" candidate_uuid: "92abc2e72186410499a20b7025ae5968" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d122b723bcfe46fd92ec072ca39d66b4" is_pre_election: true
I20260808 02:16:05.158849 10701 raft_consensus.cc:2454] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92abc2e72186410499a20b7025ae5968 in term 0.
I20260808 02:16:05.158845 10802 leader_election.cc:302] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92abc2e72186410499a20b7025ae5968, a4be58d54faf4991b3faf816f019a95e; no voters:
I20260808 02:16:05.158078 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.154009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44725 (local address 127.19.58.193:58037)
0808 02:16:05.154250 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:05.154260 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:05.154339 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:05.154945 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:05.154946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:05.154948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:05.155008 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:05.155010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.155862 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.155862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:05.156919 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:05.156922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.157611 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.157612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:05.157671 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.157674 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:05.157785 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 02:16:05.157986 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":49,"threads_started":1}
I20260808 02:16:05.158954 11141 raft_consensus.cc:2781] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:16:05.159017 11141 raft_consensus.cc:482] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:16:05.159040 11141 raft_consensus.cc:3037] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:05.159152 10895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30361401d156421c85d6744047976515" candidate_uuid: "d122b723bcfe46fd92ec072ca39d66b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92abc2e72186410499a20b7025ae5968" is_pre_election: true
I20260808 02:16:05.159859 11141 raft_consensus.cc:504] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:05.159996 10895 raft_consensus.cc:2379] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d122b723bcfe46fd92ec072ca39d66b4 in current term 1: Already voted for candidate 92abc2e72186410499a20b7025ae5968 in this term.
I20260808 02:16:05.159996 11141 leader_election.cc:288] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [CANDIDATE]: Term 1 election: Requested vote from peers a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139), d122b723bcfe46fd92ec072ca39d66b4 (127.19.58.193:37141)
I20260808 02:16:05.160161 11061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30361401d156421c85d6744047976515" candidate_uuid: "92abc2e72186410499a20b7025ae5968" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4be58d54faf4991b3faf816f019a95e"
I20260808 02:16:05.160168 10701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30361401d156421c85d6744047976515" candidate_uuid: "92abc2e72186410499a20b7025ae5968" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d122b723bcfe46fd92ec072ca39d66b4"
I20260808 02:16:05.160235 11061 raft_consensus.cc:3037] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:05.160240 10701 raft_consensus.cc:3037] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:05.160892 11061 raft_consensus.cc:2454] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92abc2e72186410499a20b7025ae5968 in term 1.
I20260808 02:16:05.160923 10701 raft_consensus.cc:2454] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92abc2e72186410499a20b7025ae5968 in term 1.
I20260808 02:16:05.161072 10802 leader_election.cc:302] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92abc2e72186410499a20b7025ae5968, d122b723bcfe46fd92ec072ca39d66b4; no voters:
I20260808 02:16:05.161098 10611 leader_election.cc:302] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d122b723bcfe46fd92ec072ca39d66b4; no voters: 92abc2e72186410499a20b7025ae5968, a4be58d54faf4991b3faf816f019a95e
I20260808 02:16:05.161161 11141 raft_consensus.cc:2781] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:05.161247 10773 raft_consensus.cc:2726] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 02:16:05.161314 11141 raft_consensus.cc:686] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 1 LEADER]: Becoming Leader. State: Replica: 92abc2e72186410499a20b7025ae5968, State: Running, Role: LEADER
I20260808 02:16:05.161406 11141 consensus_queue.cc:237] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:05.162143 10511 catalog_manager.cc:5495] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92abc2e72186410499a20b7025ae5968 (127.19.58.194). New cstate: current_term: 1 leader_uuid: "92abc2e72186410499a20b7025ae5968" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } health_report { overall_health: UNKNOWN } } }
I20260808 02:16:05.229017 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.226620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54536 (local address 127.19.58.194:44725)
0808 02:16:05.226674 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:05.226676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:05.226684 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:16:05.226750 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:05.226751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:05.226771 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:05.226830 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:05.226967 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.227492 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.228325 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.228517 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.228576 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.228724 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:05.228770 (+ 46us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 02:16:05.228929 (+ 159us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54536: token expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 02:16:05.229085 11134 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54536: token expired
I20260808 02:16:05.234761 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.229334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35648 (local address 127.19.58.254:42405)
0808 02:16:05.229378 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:05.229380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:05.229421 (+ 41us) server_negotiation.cc:407] Connection header received
0808 02:16:05.229468 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:05.229469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:05.229476 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:05.229518 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:05.229941 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.230386 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.231048 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.231238 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.231299 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.232519 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:05.232521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:05.232521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:05.232528 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:05.233971 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:05.234233 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:05.234234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:05.234234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:05.234281 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:05.234400 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:05.234401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:05.234401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:05.234487 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:05.234553 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:05.234623 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 02:16:05.234685 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:16:05.237462 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.235507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54550 (local address 127.19.58.194:44725)
0808 02:16:05.235542 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:05.235544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:05.235551 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:16:05.235606 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:05.235607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:05.235616 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:05.235658 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:05.235756 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.236190 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.236851 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:05.237029 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:05.237080 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:05.237194 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:05.237283 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:05.237325 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:05.237377 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 02:16:05.237394 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:16:05.239030 11061 raft_consensus.cc:1261] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e [term 1 FOLLOWER]: Refusing update from remote peer 92abc2e72186410499a20b7025ae5968: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:05.239050 10701 raft_consensus.cc:1261] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Refusing update from remote peer 92abc2e72186410499a20b7025ae5968: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:05.239317 11141 consensus_queue.cc:1035] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:05.239485 11150 consensus_queue.cc:1035] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:05.241436 11153 mvcc.cc:205] Tried to move back new op lower bound from 7316092376016715776 to 7316092375701602304. Current Snapshot: MvccSnapshot[applied={T|T < 7316092376016715776}]
I20260808 02:16:05.241606 11154 mvcc.cc:205] Tried to move back new op lower bound from 7316092376016715776 to 7316092375701602304. Current Snapshot: MvccSnapshot[applied={T|T < 7316092376016715776}]
I20260808 02:16:05.242410 11155 mvcc.cc:205] Tried to move back new op lower bound from 7316092376016715776 to 7316092375701602304. Current Snapshot: MvccSnapshot[applied={T|T < 7316092376016715776}]
I20260808 02:16:05.243566 10508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:05.243657 10508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:05.244369 10508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Sending DeleteTablet for 3 replicas of tablet 30361401d156421c85d6744047976515
I20260808 02:16:05.244663 10865 tablet_service.cc:1514] Processing DeleteTablet for tablet 30361401d156421c85d6744047976515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:05 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:54526
I20260808 02:16:05.244709 11038 tablet_service.cc:1514] Processing DeleteTablet for tablet 30361401d156421c85d6744047976515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:05 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:35272
I20260808 02:16:05.244740 10673 tablet_service.cc:1514] Processing DeleteTablet for tablet 30361401d156421c85d6744047976515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:05 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:33614
I20260808 02:16:05.244777 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 10567
I20260808 02:16:05.255856 11167 tablet_replica.cc:330] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: stopping tablet replica
I20260808 02:16:05.255964 11167 raft_consensus.cc:2229] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:16:05.256053 11167 raft_consensus.cc:2258] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:05.256199 11169 tablet_replica.cc:330] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: stopping tablet replica
I20260808 02:16:05.256352 11169 raft_consensus.cc:2229] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:05.256502 11169 raft_consensus.cc:2258] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:05.256559 11167 ts_tablet_manager.cc:1905] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:05.257017 11169 ts_tablet_manager.cc:1905] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 02:16:05.257469 10489 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:37141 (error 108)
I20260808 02:16:05.258179 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 10771
I20260808 02:16:05.258471 11167 ts_tablet_manager.cc:1918] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:05.258527 11167 log.cc:1196] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/30361401d156421c85d6744047976515
I20260808 02:16:05.259737 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.259596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:37141 (local address 127.0.0.1:33620)
0808 02:16:05.259645 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:05.259683 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:37141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 02:16:05.259838 10489 catalog_manager.cc:4584] TS d122b723bcfe46fd92ec072ca39d66b4 (127.19.58.193:37141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30361401d156421c85d6744047976515: Network error: Client connection negotiation failed: client connection to 127.19.58.193:37141: connect: Connection refused (error 111)
I20260808 02:16:05.260995 11169 ts_tablet_manager.cc:1918] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:05.261057 11169 log.cc:1196] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/30361401d156421c85d6744047976515
I20260808 02:16:05.261319 11169 ts_tablet_manager.cc:1939] T 30361401d156421c85d6744047976515 P a4be58d54faf4991b3faf816f019a95e: Deleting consensus metadata
I20260808 02:16:05.261698 10489 catalog_manager.cc:4841] TS a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139): tablet 30361401d156421c85d6744047976515 (table test-table [id=ef8133f4874f410f9231ecbb9fb001d1]) successfully deleted
I20260808 02:16:05.265899 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:05.265806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44725 (local address 127.0.0.1:54560)
0808 02:16:05.265850 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:05.265872 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.194:44725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 02:16:05.265913 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 10953
I20260808 02:16:05.272315 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 10460
I20260808 02:16:25.285569 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42405
--webserver_interface=127.19.58.254
--webserver_port=44575
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:42405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:25.367611 11174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:25.367857 11174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:25.367880 11174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:25.369056 11174 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 02:16:25.369100 11174 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 02:16:25.369112 11174 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 02:16:25.369123 11174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:25.369146 11174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:25.369164 11174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:25.369186 11174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:25.369201 11174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:25.370698 11174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:42405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=44575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:25.370941 11174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:25.371170 11174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:25.376307 11180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:25.376206 11181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:25.376261 11184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:25.376211 11174 server_base.cc:1034] running on GCE node
I20260808 02:16:25.376742 11174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:25.377002 11174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:25.378140 11174 hybrid_clock.cc:648] HybridClock initialized: now 1786155385378129 us; error 32 us; skew 500 ppm
Aug 08 02:16:25 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155385, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:25.386541 11174 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:16:25.386982 11174 webserver.cc:460] Webserver started at http://127.19.58.254:44575/ using document root <none> and password file <none>
I20260808 02:16:25.387194 11174 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:25.387244 11174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:25.388617 11174 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:25.389249 11193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:25.389398 11174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:25.389482 11174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "0d892d7e948f46c3991994124f464efa"
format_stamp: "Formatted at 2026-08-08 02:15:43 on dist-test-slave-1l3l"
server_key: "428c634db136a75b2e5084a8b0aba5e3"
server_key_iv: "6bacfb62d58016daab39b8419b003d4b"
server_key_version: "encryptionkey@0"
I20260808 02:16:25.389801 11174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:25.406674 11174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:25.408948 11174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:25.409093 11174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:25.413306 11174 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:42405
I20260808 02:16:25.413353 11266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:42405 every 8 connection(s)
I20260808 02:16:25.413712 11174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:25.415166 11268 sys_catalog.cc:263] Verifying existing consensus state
I20260808 02:16:25.415851 11268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Bootstrap starting.
I20260808 02:16:25.421664 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 11174
I20260808 02:16:25.421940 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:37141
--local_ip_for_outbound_sockets=127.19.58.193
--tserver_master_addrs=127.19.58.254:42405
--webserver_port=43585
--webserver_interface=127.19.58.193
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:16:25.424960 11268 log.cc:824] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:25.427045 11268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:16:25.427321 11268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: Bootstrap complete.
I20260808 02:16:25.429448 11268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:16:25.429662 11268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d892d7e948f46c3991994124f464efa, State: Initialized, Role: FOLLOWER
I20260808 02:16:25.429812 11268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:16:25.429898 11268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:25.429949 11268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:25.429996 11268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 2 FOLLOWER]: Advancing to term 3
I20260808 02:16:25.430943 11268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:16:25.431036 11268 leader_election.cc:302] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d892d7e948f46c3991994124f464efa; no voters:
I20260808 02:16:25.431205 11268 leader_election.cc:288] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 02:16:25.431253 11274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 3 FOLLOWER]: Leader election won for term 3
I20260808 02:16:25.431416 11274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [term 3 LEADER]: Becoming Leader. State: Replica: 0d892d7e948f46c3991994124f464efa, State: Running, Role: LEADER
I20260808 02:16:25.431463 11268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:25.431532 11274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } }
I20260808 02:16:25.431862 11275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0d892d7e948f46c3991994124f464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } } }
I20260808 02:16:25.431934 11275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:25.432101 11276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d892d7e948f46c3991994124f464efa. Latest consensus state: current_term: 3 leader_uuid: "0d892d7e948f46c3991994124f464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d892d7e948f46c3991994124f464efa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42405 } } }
I20260808 02:16:25.432188 11276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:25.433249 11294 catalog_manager.cc:1259] Loaded cluster ID: 7bfa009d32c646a188250dcf1e4c26cc
I20260808 02:16:25.433303 11294 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: loading cluster ID for follower catalog manager: success
I20260808 02:16:25.434442 11294 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0d892d7e948f46c3991994124f464efa: acquiring CA information for follower catalog manager: success
F20260808 02:16:25.434810 11294 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0ce28f19ad google::LogMessage::Fail() at ??:0
@ 0x7f0ce28f488a google::LogMessage::SendToLog() at ??:0
@ 0x7f0ce28f14de google::LogMessage::Flush() at ??:0
@ 0x7f0ce28f2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0ce505e104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0cea84e6b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f0cea84e989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f0cea84ee4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f0cea8518c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f0ce47b4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0ce56186db start_thread at ??:0
@ 0x7f0ce13ef61f clone at ??:0
W20260808 02:16:25.517974 11271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:25.518168 11271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:25.518190 11271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:25.519306 11271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:25.519347 11271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:25.519372 11271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:16:25.520819 11271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:37141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=43585
--tserver_master_addrs=127.19.58.254:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:25.521015 11271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:25.521226 11271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:16:25.526163 11271 server_base.cc:1034] running on GCE node
W20260808 02:16:25.526137 11320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:25.526137 11319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:25.526417 11322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:25.526634 11271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:25.526875 11271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:25.528030 11271 hybrid_clock.cc:648] HybridClock initialized: now 1786155385528014 us; error 37 us; skew 500 ppm
Aug 08 02:16:25 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155385, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:25.530586 11271 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:16:25.531046 11271 webserver.cc:460] Webserver started at http://127.19.58.193:43585/ using document root <none> and password file <none>
I20260808 02:16:25.531251 11271 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:25.531301 11271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:25.532665 11271 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:25.533295 11333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:25.533504 11271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:25.533609 11271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "d122b723bcfe46fd92ec072ca39d66b4"
format_stamp: "Formatted at 2026-08-08 02:15:43 on dist-test-slave-1l3l"
server_key: "45f1e14cbd5547fc383d82f61b73789d"
server_key_iv: "a699fa293e6ab4b6f7e7c6b6955bfe87"
server_key_version: "encryptionkey@0"
I20260808 02:16:25.534067 11271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:25.574398 11271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:25.576898 11271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:25.577060 11271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:25.577342 11271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:25.577852 11343 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 02:16:25.579311 11271 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 02:16:25.579373 11271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 02:16:25.579407 11271 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 02:16:25.580183 11271 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 02:16:25.580224 11271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 02:16:25.580318 11343 tablet_bootstrap.cc:492] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Bootstrap starting.
I20260808 02:16:25.586874 11271 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:37141
I20260808 02:16:25.586967 11490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:37141 every 8 connection(s)
I20260808 02:16:25.587328 11271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:25.587936 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 11271
I20260808 02:16:25.588295 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:44725
--local_ip_for_outbound_sockets=127.19.58.194
--tserver_master_addrs=127.19.58.254:42405
--webserver_port=33759
--webserver_interface=127.19.58.194
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:16:25.591063 11343 log.cc:824] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:25.593900 11343 tablet_bootstrap.cc:492] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 02:16:25.594225 11343 tablet_bootstrap.cc:492] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Bootstrap complete.
I20260808 02:16:25.595379 11343 ts_tablet_manager.cc:1397] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260808 02:16:25.596827 11343 raft_consensus.cc:348] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:25.597169 11343 raft_consensus.cc:729] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d122b723bcfe46fd92ec072ca39d66b4, State: Initialized, Role: FOLLOWER
I20260808 02:16:25.597349 11343 consensus_queue.cc:260] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:25.597635 11343 ts_tablet_manager.cc:1428] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4: Time spent starting tablet: real 0.002s user 0.003s sys 0.003s
W20260808 02:16:25.634500 11493 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:16:25.587969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:42263)
0808 02:16:25.590372 (+ 2403us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:25.590431 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:25.591055 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:25.634286 (+ 43231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2352,"thread_start_us":99,"threads_started":1}
W20260808 02:16:25.634753 11491 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:42405 (0 consecutive failures): Network error: Failed to ping master at 127.19.58.254:42405: Client connection negotiation failed: client connection to 127.19.58.254:42405: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 02:16:25.635103 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:25.634995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:33371)
0808 02:16:25.635041 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:25.635083 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 02:16:25.635452 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:25.635369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:56533)
0808 02:16:25.635413 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:25.635436 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 02:16:25.635565 11491 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 02:16:25.673419 11494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:25.673609 11494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:25.673630 11494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:25.674746 11494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:25.674789 11494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:25.674814 11494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:16:25.676234 11494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:44725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=33759
--tserver_master_addrs=127.19.58.254:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:25.676406 11494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:25.676576 11494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:25.681166 11504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:25.681177 11507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:25.681363 11494 server_base.cc:1034] running on GCE node
W20260808 02:16:25.681440 11503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:25.681612 11494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:25.681802 11494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:25.682929 11494 hybrid_clock.cc:648] HybridClock initialized: now 1786155385682920 us; error 29 us; skew 500 ppm
Aug 08 02:16:25 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155385, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:25.685346 11494 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:16:25.685793 11494 webserver.cc:460] Webserver started at http://127.19.58.194:33759/ using document root <none> and password file <none>
I20260808 02:16:25.685992 11494 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:25.686043 11494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:25.687321 11494 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:25.688046 11518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:25.688216 11494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:16:25.688301 11494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "92abc2e72186410499a20b7025ae5968"
format_stamp: "Formatted at 2026-08-08 02:15:44 on dist-test-slave-1l3l"
server_key: "bfcf5e97ff421d38ff6dac23dcefaa65"
server_key_iv: "6d6e6d1aadcc023ab99e2e2a13e0b4b5"
server_key_version: "encryptionkey@0"
I20260808 02:16:25.688637 11494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:25.699343 11494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:25.701097 11494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:25.701229 11494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:25.701442 11494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:25.701824 11528 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260808 02:16:25.703157 11528 ts_tablet_manager.cc:1852] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260808 02:16:25.703420 11528 ts_tablet_manager.cc:1905] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:25.705518 11528 ts_tablet_manager.cc:1918] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:25.705592 11528 log.cc:1196] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/30361401d156421c85d6744047976515
I20260808 02:16:25.705817 11528 ts_tablet_manager.cc:1939] T 30361401d156421c85d6744047976515 P 92abc2e72186410499a20b7025ae5968: Deleting consensus metadata
I20260808 02:16:25.705981 11494 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260808 02:16:25.706027 11494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 02:16:25.706066 11494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:25.706091 11494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:25.711869 11494 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:44725
I20260808 02:16:25.711961 11675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:44725 every 8 connection(s)
I20260808 02:16:25.712193 11494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:16:25.713128 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:25.712783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:58025)
0808 02:16:25.712938 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:25.713002 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
W20260808 02:16:25.713341 11676 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:42405 (0 consecutive failures): Network error: Failed to ping master at 127.19.58.254:42405: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
I20260808 02:16:25.713594 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:25.713512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:45237)
0808 02:16:25.713551 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:25.713576 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 02:16:25.713802 11678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:25.713737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:47591)
0808 02:16:25.713770 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:25.713789 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 02:16:25.713876 11676 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:16:25.714529 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 11494
I20260808 02:16:25.714773 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:46139
--local_ip_for_outbound_sockets=127.19.58.195
--tserver_master_addrs=127.19.58.254:42405
--webserver_port=35983
--webserver_interface=127.19.58.195
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:25.794982 11679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:25.795176 11679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:25.795197 11679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:25.796320 11679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:25.796357 11679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:25.796389 11679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:16:25.797736 11679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:46139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=35983
--tserver_master_addrs=127.19.58.254:42405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:25.797905 11679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:25.798076 11679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:25.802817 11688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:25.802939 11685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:25.803018 11679 server_base.cc:1034] running on GCE node
W20260808 02:16:25.802824 11686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:25.803288 11679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:25.803501 11679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:25.804634 11679 hybrid_clock.cc:648] HybridClock initialized: now 1786155385804614 us; error 37 us; skew 500 ppm
Aug 08 02:16:25 dist-test-slave-1l3l krb5kdc[9728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155385, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:25.807142 11679 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:16:25.807576 11679 webserver.cc:460] Webserver started at http://127.19.58.195:35983/ using document root <none> and password file <none>
I20260808 02:16:25.807791 11679 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:25.807840 11679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:25.809154 11679 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:16:25.809764 11697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:25.809940 11679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:16:25.810024 11679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "a4be58d54faf4991b3faf816f019a95e"
format_stamp: "Formatted at 2026-08-08 02:15:44 on dist-test-slave-1l3l"
server_key: "b07c4c8d869970df8525f75b59887108"
server_key_iv: "0c84b3a9ab57a80f44874b431faffbb4"
server_key_version: "encryptionkey@0"
I20260808 02:16:25.810343 11679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:25.824261 11679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:25.826226 11679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:25.826333 11679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:25.826511 11679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:25.826745 11679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:25.826773 11679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:25.826792 11679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:25.826803 11679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:25.832226 11679 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:46139
I20260808 02:16:25.832285 11852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:46139 every 8 connection(s)
I20260808 02:16:25.832589 11679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:16:25.833509 11856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:25.833168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:40637)
0808 02:16:25.833302 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:25.833368 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
W20260808 02:16:25.833858 11853 heartbeater.cc:644] Failed to heartbeat to 127.19.58.254:42405 (0 consecutive failures): Network error: Failed to ping master at 127.19.58.254:42405: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
I20260808 02:16:25.834108 11856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:25.834030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:58165)
0808 02:16:25.834067 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:25.834092 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 02:16:25.834326 11856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:25.834266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:51541)
0808 02:16:25.834296 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:25.834314 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 02:16:25.834406 11853 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 02:16:25.839995 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 11679
I20260808 02:16:26.636152 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:26.635928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:59361)
0808 02:16:26.636055 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:26.636085 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260808 02:16:26.714350 11862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:26.714165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:55793)
0808 02:16:26.714280 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:26.714314 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260808 02:16:26.752354 11863 raft_consensus.cc:482] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:26.752532 11863 raft_consensus.cc:504] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:26.752928 11863 leader_election.cc:288] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725), a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139)
Aug 08 02:16:26 dist-test-slave-1l3l krb5kdc[9728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155385, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.195@KRBTEST.COM
Aug 08 02:16:26 dist-test-slave-1l3l krb5kdc[9728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155385, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.194@KRBTEST.COM
I20260808 02:16:26.764442 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:26.753049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:44725 (local address 127.19.58.193:48743)
0808 02:16:26.753090 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:26.753102 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:26.753164 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:26.754169 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:26.754170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:26.754512 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:26.754777 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:26.754785 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:26.756144 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:26.756145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:26.757138 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:26.757141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:26.757964 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:26.757965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:26.758102 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:26.758634 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:26.758645 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:26.760881 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:26.762758 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:26.762765 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:26.762767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:26.762981 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:26.763253 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:26.763254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:26.763255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:26.763399 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:26.763886 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:26.763893 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:26.764061 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 02:16:26.764177 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 02:16:26.764335 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 02:16:26.764497 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:26.753157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:48743 (local address 127.19.58.194:44725)
0808 02:16:26.753351 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:26.753362 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:26.753388 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:16:26.753994 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:26.753997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:26.754045 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:26.754179 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:26.754897 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:26.756048 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:26.757239 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:26.757848 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:26.758387 (+ 539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:26.760985 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:26.760989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:26.760989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:26.761001 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:26.762679 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:26.763054 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:26.763057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:26.763058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:26.763167 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:26.763477 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:26.763478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:26.763478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:26.763814 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:26.763906 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:26.764246 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 02:16:26.764383 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":66,"threads_started":1}
I20260808 02:16:26.764979 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:26.753189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:49257 (local address 127.19.58.195:46139)
0808 02:16:26.753350 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:26.753363 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:26.753387 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:16:26.753811 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:26.753814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:26.753874 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:26.753999 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:26.754932 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:26.756855 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:26.757755 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:26.758272 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:26.758380 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:26.760610 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:26.760615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:26.760615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:26.760967 (+ 352us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:26.763158 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:26.763591 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:26.763595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:26.763595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:26.763735 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:26.763943 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:26.763944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:26.763944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:26.764246 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:26.764331 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:26.764769 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 02:16:26.764894 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260808 02:16:26.764658 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:26.753054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:46139 (local address 127.19.58.193:49257)
0808 02:16:26.753280 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:26.753289 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:26.753329 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:26.754052 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:26.754055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:26.754511 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:26.754778 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:26.754785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:26.756980 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:26.756981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:26.757650 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:26.757654 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:26.758345 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:26.758346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:26.758411 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:26.758744 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:26.758747 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:26.760483 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:26.763257 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:26.763258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:26.763259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:26.763509 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:26.763792 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:26.763793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:26.763793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:26.763871 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:26.764363 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:26.764364 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:26.764405 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:16:26.764466 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 02:16:26.764552 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":140,"thread_start_us":51,"threads_started":1}
W20260808 02:16:26.767544 11336 leader_election.cc:341] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725): Not found: Tablet not found: 30361401d156421c85d6744047976515
W20260808 02:16:26.767699 11335 leader_election.cc:341] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139): Not found: Tablet not found: 30361401d156421c85d6744047976515
I20260808 02:16:26.767778 11335 leader_election.cc:302] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d122b723bcfe46fd92ec072ca39d66b4; no voters: 92abc2e72186410499a20b7025ae5968, a4be58d54faf4991b3faf816f019a95e
I20260808 02:16:26.767915 11863 raft_consensus.cc:2726] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:16:26.834913 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:26.834704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:36251)
0808 02:16:26.834826 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:26.834874 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260808 02:16:27.636816 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:27.636575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:44173)
0808 02:16:27.636729 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:27.636767 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260808 02:16:27.714979 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:27.714743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:42567)
0808 02:16:27.714898 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:27.714939 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
I20260808 02:16:27.835525 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:27.835321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:39283)
0808 02:16:27.835449 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:27.835485 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260808 02:16:28.637480 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:28.637273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:40061)
0808 02:16:28.637397 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:28.637432 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260808 02:16:28.715617 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:28.715388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:55225)
0808 02:16:28.715537 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:28.715579 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260808 02:16:28.836102 11880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:28.835910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:37227)
0808 02:16:28.836031 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:28.836063 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260808 02:16:28.898705 11887 raft_consensus.cc:482] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:28.898805 11887 raft_consensus.cc:504] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:28.898979 11887 leader_election.cc:288] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725), a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139)
W20260808 02:16:28.899408 11336 leader_election.cc:341] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725): Not found: Tablet not found: 30361401d156421c85d6744047976515
W20260808 02:16:28.899502 11335 leader_election.cc:341] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139): Not found: Tablet not found: 30361401d156421c85d6744047976515
I20260808 02:16:28.899551 11335 leader_election.cc:302] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d122b723bcfe46fd92ec072ca39d66b4; no voters: 92abc2e72186410499a20b7025ae5968, a4be58d54faf4991b3faf816f019a95e
I20260808 02:16:28.899614 11887 raft_consensus.cc:2726] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:16:29.638145 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:29.637909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:44909)
0808 02:16:29.638063 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:29.638098 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260808 02:16:29.716254 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:29.716048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:34501)
0808 02:16:29.716177 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:29.716213 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260808 02:16:29.836720 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:29.836490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:49121)
0808 02:16:29.836643 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:29.836681 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260808 02:16:30.638907 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:30.638684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:57505)
0808 02:16:30.638824 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:30.638858 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260808 02:16:30.716863 11893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:30.716652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:54403)
0808 02:16:30.716787 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:30.716822 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260808 02:16:30.837466 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:30.837174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:54741)
0808 02:16:30.837375 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:30.837424 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":75,"threads_started":1}
I20260808 02:16:31.150327 11896 raft_consensus.cc:482] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:31.150439 11896 raft_consensus.cc:504] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:31.150612 11896 leader_election.cc:288] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725), a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139)
W20260808 02:16:31.151034 11336 leader_election.cc:341] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725): Not found: Tablet not found: 30361401d156421c85d6744047976515
W20260808 02:16:31.151134 11335 leader_election.cc:341] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139): Not found: Tablet not found: 30361401d156421c85d6744047976515
I20260808 02:16:31.151170 11335 leader_election.cc:302] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d122b723bcfe46fd92ec072ca39d66b4; no voters: 92abc2e72186410499a20b7025ae5968, a4be58d54faf4991b3faf816f019a95e
I20260808 02:16:31.151243 11896 raft_consensus.cc:2726] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:16:31.639580 11897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:31.639326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:59005)
0808 02:16:31.639491 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:31.639530 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":109,"threads_started":1}
I20260808 02:16:31.717525 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:31.717304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:58091)
0808 02:16:31.717447 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:31.717483 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260808 02:16:31.838217 11899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:31.837989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:35143)
0808 02:16:31.838124 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:31.838164 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260808 02:16:32.640210 11900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:32.640005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:52267)
0808 02:16:32.640125 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:32.640161 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260808 02:16:32.718133 11902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:32.717911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:35203)
0808 02:16:32.718054 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:32.718090 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260808 02:16:32.838984 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:32.838656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:51937)
0808 02:16:32.838902 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:32.838941 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":70,"threads_started":1}
I20260808 02:16:33.640831 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:33.640605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:58937)
0808 02:16:33.640740 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:33.640778 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260808 02:16:33.718694 11905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:33.718505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:56607)
0808 02:16:33.718621 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:33.718655 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260808 02:16:33.839609 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:33.839357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:53003)
0808 02:16:33.839528 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:33.839567 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260808 02:16:34.641505 11907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:34.641232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:53855)
0808 02:16:34.641414 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:34.641454 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":89,"threads_started":1}
I20260808 02:16:34.719343 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:34.719124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:52997)
0808 02:16:34.719267 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:34.719303 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260808 02:16:34.840231 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:34.840006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:51335)
0808 02:16:34.840152 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:34.840190 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260808 02:16:35.139073 11911 raft_consensus.cc:482] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:35.139179 11911 raft_consensus.cc:504] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:35.139339 11911 leader_election.cc:288] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725), a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139)
W20260808 02:16:35.139755 11336 leader_election.cc:341] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725): Not found: Tablet not found: 30361401d156421c85d6744047976515
W20260808 02:16:35.139853 11335 leader_election.cc:341] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139): Not found: Tablet not found: 30361401d156421c85d6744047976515
I20260808 02:16:35.139899 11335 leader_election.cc:302] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d122b723bcfe46fd92ec072ca39d66b4; no voters: 92abc2e72186410499a20b7025ae5968, a4be58d54faf4991b3faf816f019a95e
I20260808 02:16:35.139987 11911 raft_consensus.cc:2726] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:16:35.642146 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:35.641926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:44389)
0808 02:16:35.642067 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:35.642099 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260808 02:16:35.719995 11916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:35.719798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:34055)
0808 02:16:35.719917 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:35.719953 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260808 02:16:35.840940 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:35.840622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:57803)
0808 02:16:35.840793 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:35.840895 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260808 02:16:36.642807 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:36.642587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:51169)
0808 02:16:36.642714 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:36.642750 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260808 02:16:36.720587 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:36.720386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:58281)
0808 02:16:36.720505 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:36.720544 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260808 02:16:36.841686 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:36.841433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:33043)
0808 02:16:36.841573 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:36.841618 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260808 02:16:37.643481 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:37.643232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:45825)
0808 02:16:37.643391 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:37.643431 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":68,"threads_started":1}
I20260808 02:16:37.721266 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:37.721029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:46041)
0808 02:16:37.721161 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:37.721208 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260808 02:16:37.842351 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:37.842137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:34661)
0808 02:16:37.842271 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:37.842307 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260808 02:16:38.644129 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:38.643899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:41901)
0808 02:16:38.644034 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:38.644072 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260808 02:16:38.721868 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:38.721667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:40281)
0808 02:16:38.721789 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:38.721824 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260808 02:16:38.843006 11927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:38.842786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:59519)
0808 02:16:38.842923 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:38.842961 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260808 02:16:39.644774 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.644583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:56599)
0808 02:16:39.644691 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.644726 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":53,"threads_started":1}
I20260808 02:16:39.722436 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.722234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:56855)
0808 02:16:39.722356 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.722388 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260808 02:16:39.843742 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:39.843464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:37367)
0808 02:16:39.843617 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:39.843666 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260808 02:16:40.372786 11932 raft_consensus.cc:482] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:40.372889 11932 raft_consensus.cc:504] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92abc2e72186410499a20b7025ae5968" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 44725 } } peers { permanent_uuid: "a4be58d54faf4991b3faf816f019a95e" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46139 } } peers { permanent_uuid: "d122b723bcfe46fd92ec072ca39d66b4" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37141 } }
I20260808 02:16:40.373046 11932 leader_election.cc:288] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725), a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139)
W20260808 02:16:40.373427 11336 leader_election.cc:341] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92abc2e72186410499a20b7025ae5968 (127.19.58.194:44725): Not found: Tablet not found: 30361401d156421c85d6744047976515
W20260808 02:16:40.373515 11335 leader_election.cc:341] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a4be58d54faf4991b3faf816f019a95e (127.19.58.195:46139): Not found: Tablet not found: 30361401d156421c85d6744047976515
I20260808 02:16:40.373582 11335 leader_election.cc:302] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d122b723bcfe46fd92ec072ca39d66b4; no voters: 92abc2e72186410499a20b7025ae5968, a4be58d54faf4991b3faf816f019a95e
I20260808 02:16:40.373651 11932 raft_consensus.cc:2726] T 30361401d156421c85d6744047976515 P d122b723bcfe46fd92ec072ca39d66b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 02:16:40.645547 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:40.645304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.193:54855)
0808 02:16:40.645449 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:40.645490 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260808 02:16:40.723111 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:40.722906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.194:41749)
0808 02:16:40.723034 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:40.723068 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260808 02:16:40.844477 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:40.844212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42405 (local address 127.19.58.195:33203)
0808 02:16:40.844385 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:40.844426 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.254:42405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":65,"threads_started":1}
I20260808 02:16:40.845328 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 11271
I20260808 02:16:40.856762 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 11494
I20260808 02:16:40.866371 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 11679
2026-08-08T02:16:40Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58497 ms)
[----------] 30 tests from SecurityITest (230335 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:40 dist-test-slave-1l3l krb5kdc[11938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:40 dist-test-slave-1l3l krb5kdc[11938](info): set up 2 sockets
Aug 08 02:16:40 dist-test-slave-1l3l krb5kdc[11938](info): commencing operation
krb5kdc: starting...
W20260808 02:16:41.999842 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:16:42 dist-test-slave-1l3l krb5kdc[11938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:16:42.116616 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:38015
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:38015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:42.198031 11959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:42.198239 11959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:42.198268 11959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:42.199397 11959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:42.199440 11959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:42.199457 11959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:42.199474 11959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:42.199491 11959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:42.200985 11959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:38015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:38015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:42.201231 11959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:42.201438 11959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:42.206132 11964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:42.206221 11967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:42.206279 11959 server_base.cc:1034] running on GCE node
W20260808 02:16:42.206218 11965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:42.206691 11959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:42.206982 11959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:42.208132 11959 hybrid_clock.cc:648] HybridClock initialized: now 1786155402208122 us; error 29 us; skew 500 ppm
Aug 08 02:16:42 dist-test-slave-1l3l krb5kdc[11938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:42.214958 11959 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:16:42.215405 11959 webserver.cc:460] Webserver started at http://127.19.58.254:45987/ using document root <none> and password file <none>
I20260808 02:16:42.215602 11959 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:42.215655 11959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:42.215801 11959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:42.216801 11959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "35129b963ae4487c89b9569e25f2aba9"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "c65910b54c487c301ba0a6bb8b8a28b5"
server_key_iv: "1468fb94028cb9fb26ccce0d95d3e83b"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.217113 11959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "35129b963ae4487c89b9569e25f2aba9"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "c65910b54c487c301ba0a6bb8b8a28b5"
server_key_iv: "1468fb94028cb9fb26ccce0d95d3e83b"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.218362 11959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:16:42.219009 11981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:42.219148 11959 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:42.219238 11959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "35129b963ae4487c89b9569e25f2aba9"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "c65910b54c487c301ba0a6bb8b8a28b5"
server_key_iv: "1468fb94028cb9fb26ccce0d95d3e83b"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.219300 11959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:42.233402 11959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:42.235204 11959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:42.235353 11959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:42.239308 11959 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:38015
I20260808 02:16:42.239357 12050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:38015 every 8 connection(s)
I20260808 02:16:42.239723 11959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:42.240602 12051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:42.241228 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 11959
I20260808 02:16:42.241312 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:42.241489 19691 external_mini_cluster.cc:1428] Setting key ec733a9f6662561a318a8c91a1a0029f
I20260808 02:16:42.243221 12051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9: Bootstrap starting.
I20260808 02:16:42.244001 12051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:42.244316 12051 log.cc:824] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:42.244982 12051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9: No bootstrap required, opened a new log
I20260808 02:16:42.247153 12051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35129b963ae4487c89b9569e25f2aba9" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38015 } }
I20260808 02:16:42.247283 12051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:42.247318 12051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35129b963ae4487c89b9569e25f2aba9, State: Initialized, Role: FOLLOWER
I20260808 02:16:42.247447 12051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35129b963ae4487c89b9569e25f2aba9" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38015 } }
I20260808 02:16:42.247520 12051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:42.247551 12051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:42.247598 12051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 02:16:42 dist-test-slave-1l3l krb5kdc[11938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:42.248178 12051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35129b963ae4487c89b9569e25f2aba9" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38015 } }
I20260808 02:16:42.248272 12051 leader_election.cc:302] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35129b963ae4487c89b9569e25f2aba9; no voters:
I20260808 02:16:42.248432 12051 leader_election.cc:288] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:42.248474 12059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:42.248623 12059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [term 1 LEADER]: Becoming Leader. State: Replica: 35129b963ae4487c89b9569e25f2aba9, State: Running, Role: LEADER
I20260808 02:16:42.248731 12051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:42.248750 12059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35129b963ae4487c89b9569e25f2aba9" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38015 } }
I20260808 02:16:42.249135 12061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35129b963ae4487c89b9569e25f2aba9. Latest consensus state: current_term: 1 leader_uuid: "35129b963ae4487c89b9569e25f2aba9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35129b963ae4487c89b9569e25f2aba9" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38015 } } }
I20260808 02:16:42.249131 12060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35129b963ae4487c89b9569e25f2aba9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35129b963ae4487c89b9569e25f2aba9" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38015 } } }
I20260808 02:16:42.249236 12061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:42.249239 12060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:42.249926 12066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:42.250396 12066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:42.251940 12066 catalog_manager.cc:1347] Generated new cluster ID: f37b201f9a684f1991c5114d4b69655e
I20260808 02:16:42.252010 12066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:42.252159 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.241959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.19.58.254:38015)
0808 02:16:42.242131 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.242139 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.242163 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:16:42.242598 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.242605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.242651 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.242767 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:42.243405 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.244693 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.245488 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.246100 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.246221 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.248418 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:42.248421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:42.248422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:42.248432 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:42.250399 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:42.250791 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:42.250794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:42.250794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:42.250889 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:42.251211 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:42.251212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:42.251212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:42.251487 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:42.251631 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.251779 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.251923 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260808 02:16:42.267277 12066 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:42.267841 12066 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:42.272475 12066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9: Generated new TSK 0
I20260808 02:16:42.272635 12066 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:16:42.301846 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38015
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:42.382505 12085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:42.382700 12085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:42.382721 12085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:42.383860 12085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:42.383901 12085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:42.383926 12085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:16:42.385401 12085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:42.385636 12085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:42.385859 12085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:42.390983 12093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:42.390956 12095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:42.390938 12091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:42.391005 12085 server_base.cc:1034] running on GCE node
I20260808 02:16:42.391288 12085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:42.391499 12085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:42.392638 12085 hybrid_clock.cc:648] HybridClock initialized: now 1786155402392621 us; error 46 us; skew 500 ppm
Aug 08 02:16:42 dist-test-slave-1l3l krb5kdc[11938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:42.395165 12085 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:16:42.395623 12085 webserver.cc:460] Webserver started at http://127.19.58.193:36837/ using document root <none> and password file <none>
I20260808 02:16:42.395880 12085 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:42.395929 12085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:42.396042 12085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:42.397056 12085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "dde7892fe93e409584a25894b720ae34"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "596b12acb9056eb70f9c36e5adc34d21"
server_key_iv: "3c7f9159d2138d0ffc98f48677c10dc2"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.397362 12085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "dde7892fe93e409584a25894b720ae34"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "596b12acb9056eb70f9c36e5adc34d21"
server_key_iv: "3c7f9159d2138d0ffc98f48677c10dc2"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.398581 12085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:16:42.399241 12105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:42.399410 12085 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:42.399494 12085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "dde7892fe93e409584a25894b720ae34"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "596b12acb9056eb70f9c36e5adc34d21"
server_key_iv: "3c7f9159d2138d0ffc98f48677c10dc2"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.399549 12085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:42.406060 12085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:42.408085 12085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:42.408191 12085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:42.408351 12085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:42.408630 12085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:42.408663 12085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:42.408682 12085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:42.408705 12085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:42.414122 12085 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:38317
I20260808 02:16:42.414183 12259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:38317 every 8 connection(s)
I20260808 02:16:42.414448 12085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:42.416491 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 12085
I20260808 02:16:42.416599 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:16:42.416787 19691 external_mini_cluster.cc:1428] Setting key 73413886932f449d25b61ccf87e9670b
Aug 08 02:16:42 dist-test-slave-1l3l krb5kdc[11938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:42.423585 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.415086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:52033 (local address 127.19.58.254:38015)
0808 02:16:42.415128 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.415130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.415326 (+ 196us) server_negotiation.cc:407] Connection header received
0808 02:16:42.415893 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.415894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.415907 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.415958 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:42.416881 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.417416 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.418371 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.418582 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.418638 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.420700 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:42.420701 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:42.420702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:42.420709 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:42.422182 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:42.422645 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:42.422646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:42.422647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:42.422714 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:42.422963 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:42.422964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:42.422965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:42.423087 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:42.423149 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.423397 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.423490 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:16:42.423749 12262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.415038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:38015 (local address 127.19.58.193:52033)
0808 02:16:42.415204 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.415240 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:42.415787 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:42.416022 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:42.416026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:42.416561 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:42.416785 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.416790 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.417500 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.417501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.418289 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.418291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.418652 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.418653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.418784 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.419241 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:42.419249 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:42.420609 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:42.422295 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:42.422303 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:42.422305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:42.422575 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:42.422776 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:42.422776 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:42.422776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:42.422869 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:42.423145 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:42.423153 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:42.423274 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 02:16:42.423360 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 02:16:42.423512 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260808 02:16:42.424228 12260 heartbeater.cc:347] Connected to a master server at 127.19.58.254:38015
I20260808 02:16:42.424316 12260 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:42.424500 12260 heartbeater.cc:511] Master 127.19.58.254:38015 requested a full tablet report, sending...
I20260808 02:16:42.424993 12002 ts_manager.cc:194] Registered new tserver with Master: dde7892fe93e409584a25894b720ae34 (127.19.58.193:38317)
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
I20260808 02:16:42.425609 12002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:52033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:16:42.442663 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38015
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:42.523379 12267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:42.523571 12267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:42.523602 12267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:42.524716 12267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:42.524756 12267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:42.524791 12267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:16:42.526173 12267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:42.526412 12267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:42.526607 12267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:42.531387 12273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:42.531461 12277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:42.531379 12272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:42.531476 12267 server_base.cc:1034] running on GCE node
I20260808 02:16:42.531773 12267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:42.531966 12267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:42.533098 12267 hybrid_clock.cc:648] HybridClock initialized: now 1786155402533086 us; error 30 us; skew 500 ppm
Aug 08 02:16:42 dist-test-slave-1l3l krb5kdc[11938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:42.535414 12267 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:16:42.535856 12267 webserver.cc:460] Webserver started at http://127.19.58.194:43013/ using document root <none> and password file <none>
I20260808 02:16:42.536065 12267 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:42.536110 12267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:42.536229 12267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:42.537286 12267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "796bc3da5efd492781b4d2deaff3df12"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "e1494dfae507502e1a0845c79bfa000d"
server_key_iv: "1185b5a5577f254fa1cf054a4311e782"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.537680 12267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "796bc3da5efd492781b4d2deaff3df12"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "e1494dfae507502e1a0845c79bfa000d"
server_key_iv: "1185b5a5577f254fa1cf054a4311e782"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.539363 12267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:42.540263 12286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:42.540504 12267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 02:16:42.540603 12267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "796bc3da5efd492781b4d2deaff3df12"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "e1494dfae507502e1a0845c79bfa000d"
server_key_iv: "1185b5a5577f254fa1cf054a4311e782"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.540673 12267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:42.561398 12267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:42.563299 12267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:42.563422 12267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:42.563608 12267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:42.563923 12267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:42.563958 12267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:42.563980 12267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:42.563992 12267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:42.569350 12267 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:43527
I20260808 02:16:42.569427 12449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:43527 every 8 connection(s)
I20260808 02:16:42.569695 12267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
Aug 08 02:16:42 dist-test-slave-1l3l krb5kdc[11938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:42.577775 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.570409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:41903 (local address 127.19.58.254:38015)
0808 02:16:42.570470 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.570473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.570622 (+ 149us) server_negotiation.cc:407] Connection header received
0808 02:16:42.571101 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.571102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.571113 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.571172 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:42.572069 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.572601 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.573520 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.573715 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.573768 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.575600 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:42.575601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:42.575602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:42.575609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:42.576692 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:42.577015 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:42.577015 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:42.577016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:42.577059 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:42.577212 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:42.577212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:42.577212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:42.577300 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:42.577389 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.577639 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.577710 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 02:16:42.577836 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.570343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:38015 (local address 127.19.58.194:41903)
0808 02:16:42.570511 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.570545 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:42.570997 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:42.571223 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:42.571226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:42.571797 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:42.571990 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.571995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.572668 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.572668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.573445 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.573447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.573775 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.573775 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.573863 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.574227 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:42.574234 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:42.575522 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:42.576762 (+ 1240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:42.576768 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:42.576769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:42.576962 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:42.577095 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:42.577095 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:42.577095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:42.577175 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:42.577400 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:42.577403 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:42.577503 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 02:16:42.577580 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 02:16:42.577686 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260808 02:16:42.578251 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 12267
I20260808 02:16:42.578361 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:16:42.578361 12450 heartbeater.cc:347] Connected to a master server at 127.19.58.254:38015
I20260808 02:16:42.578461 12450 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:42.578593 19691 external_mini_cluster.cc:1428] Setting key cb6367d0cf2d7a0430226fedb1d02a27
I20260808 02:16:42.578634 12450 heartbeater.cc:511] Master 127.19.58.254:38015 requested a full tablet report, sending...
I20260808 02:16:42.579030 12001 ts_manager.cc:194] Registered new tserver with Master: 796bc3da5efd492781b4d2deaff3df12 (127.19.58.194:43527)
I20260808 02:16:42.579406 12001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:41903
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:16:42.604439 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38015
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:42.685901 12461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:42.686100 12461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:42.686121 12461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:42.687191 12461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:42.687222 12461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:42.687247 12461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:16:42.688711 12461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:38015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:42.688898 12461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:42.689083 12461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:42.693809 12468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:42.693786 12471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:42.693795 12467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:42.694002 12461 server_base.cc:1034] running on GCE node
I20260808 02:16:42.694224 12461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:42.694448 12461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:42.695586 12461 hybrid_clock.cc:648] HybridClock initialized: now 1786155402695568 us; error 36 us; skew 500 ppm
Aug 08 02:16:42 dist-test-slave-1l3l krb5kdc[11938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:42.697908 12461 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:16:42.698395 12461 webserver.cc:460] Webserver started at http://127.19.58.195:37787/ using document root <none> and password file <none>
I20260808 02:16:42.698606 12461 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:42.698655 12461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:42.698769 12461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:42.699790 12461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "9d10adaec6984153b431636987966f9a"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "6b2de55ddaa471179c752981af6787ad"
server_key_iv: "f486493da1758b5ae9c8b91191106b6f"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.700110 12461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "9d10adaec6984153b431636987966f9a"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "6b2de55ddaa471179c752981af6787ad"
server_key_iv: "f486493da1758b5ae9c8b91191106b6f"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.701359 12461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:42.702010 12479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:42.702142 12461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:42.702224 12461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "9d10adaec6984153b431636987966f9a"
format_stamp: "Formatted at 2026-08-08 02:16:42 on dist-test-slave-1l3l"
server_key: "6b2de55ddaa471179c752981af6787ad"
server_key_iv: "f486493da1758b5ae9c8b91191106b6f"
server_key_version: "encryptionkey@0"
I20260808 02:16:42.702281 12461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:42.721823 12461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:42.723848 12461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:42.723984 12461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:42.724195 12461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:42.724501 12461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:42.724535 12461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:42.724563 12461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:42.724588 12461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:42.730410 12461 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:35505
I20260808 02:16:42.730468 12639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:35505 every 8 connection(s)
I20260808 02:16:42.730748 12461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
Aug 08 02:16:42 dist-test-slave-1l3l krb5kdc[11938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155402, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:42.738889 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.731393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:37223 (local address 127.19.58.254:38015)
0808 02:16:42.731437 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.731440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.731641 (+ 201us) server_negotiation.cc:407] Connection header received
0808 02:16:42.732123 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.732124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.732133 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.732184 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:42.733087 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.733619 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.734551 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.734759 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.734817 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.736760 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:42.736761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:42.736761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:42.736767 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:42.737839 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:42.738177 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:42.738178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:42.738178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:42.738224 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:42.738432 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:42.738432 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:42.738432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:42.738525 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:42.738571 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.738769 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.738827 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:16:42.739061 12642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.731372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:38015 (local address 127.19.58.195:37223)
0808 02:16:42.731533 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.731566 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:42.732042 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:42.732246 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:42.732249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:42.732811 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:42.733008 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.733013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.733686 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.733686 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.734465 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.734467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.734827 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.734828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.734915 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.735329 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:42.735336 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:42.736675 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:42.737907 (+ 1232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:42.737913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:42.737914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:42.738120 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:42.738273 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:42.738274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:42.738274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:42.738368 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:42.738569 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:42.738573 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:42.738676 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 02:16:42.738750 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:16:42.738860 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260808 02:16:42.739476 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 12461
I20260808 02:16:42.739542 12640 heartbeater.cc:347] Connected to a master server at 127.19.58.254:38015
I20260808 02:16:42.739598 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:16:42.739642 12640 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:42.739805 19691 external_mini_cluster.cc:1428] Setting key 4107cf77f08e5b3db65f03ab854dad87
I20260808 02:16:42.739842 12640 heartbeater.cc:511] Master 127.19.58.254:38015 requested a full tablet report, sending...
I20260808 02:16:42.740216 12001 ts_manager.cc:194] Registered new tserver with Master: 9d10adaec6984153b431636987966f9a (127.19.58.195:35505)
I20260808 02:16:42.740693 12001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:37223
I20260808 02:16:42.740721 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:16:42.747916 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.742112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39310 (local address 127.19.58.254:38015)
0808 02:16:42.742168 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.742170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.742281 (+ 111us) server_negotiation.cc:407] Connection header received
0808 02:16:42.742341 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.742342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.742351 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.742396 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:42.742923 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.743379 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.744128 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.744346 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.744400 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.745522 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:42.745524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:42.745524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:42.745531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:42.746826 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:42.747134 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:42.747135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:42.747136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:42.747191 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:42.747358 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:42.747359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:42.747359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:42.747511 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:42.747601 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.747731 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.747814 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 02:16:42.749775 12001 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:16:42.751451 12001 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:16:42.757236 12653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.753437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:38317 (local address 127.0.0.1:58456)
0808 02:16:42.753666 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.753676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:42.753731 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:42.754014 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:42.754017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:42.754020 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:42.754084 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.754086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.755055 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.755056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.755986 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.755988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.756965 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.756966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.757040 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.757053 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 02:16:42.757112 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:16:42.757163 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":59,"threads_started":1}
I20260808 02:16:42.757251 12661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.753678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58456 (local address 127.19.58.193:38317)
0808 02:16:42.753856 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.753860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.753871 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:16:42.753912 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.753914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.753927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.753996 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:42.754149 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.754964 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.756180 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.756901 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.756958 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.757019 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.757133 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.757174 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260808 02:16:42.757984 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.753885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.19.58.194:43527)
0808 02:16:42.754033 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.754038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.754048 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:16:42.754087 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.754089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.754105 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.754187 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:42.754432 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.755681 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.756818 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.757512 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.757593 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.757661 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.757864 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.757910 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260808 02:16:42.757966 12655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.753422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:35505 (local address 127.0.0.1:51618)
0808 02:16:42.753543 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.753585 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:42.753647 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:42.754238 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:42.754240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:42.754242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:42.754295 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.754296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.755783 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.755784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.756845 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.756848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.757694 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.757695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.757766 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.757769 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:42.757819 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 02:16:42.757858 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260808 02:16:42.758023 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.753438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:43527 (local address 127.0.0.1:34028)
0808 02:16:42.753777 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.753788 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:42.753834 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:42.754264 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:42.754264 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:42.754266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:42.754310 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.754311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.755816 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.755817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.756719 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.756722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.757701 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.757702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.757776 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.757781 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:16:42.757847 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 02:16:42.757890 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"spinlock_wait_cycles":768,"thread_start_us":63,"threads_started":1}
I20260808 02:16:42.758279 12660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.753588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.19.58.195:35505)
0808 02:16:42.754080 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.754084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.754092 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:16:42.754124 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.754127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.754140 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.754369 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:42.754431 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.755637 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.756940 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.757626 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.757690 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.757745 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.758149 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.758197 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":110,"threads_started":1}
I20260808 02:16:42.758229 12171 tablet_service.cc:1467] Processing CreateTablet for tablet d517dc8327d04c85a30e8e86461d19da (DEFAULT_TABLE table=test-table [id=793c68207a5143c6828165acf6ef7dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:42.758572 12171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d517dc8327d04c85a30e8e86461d19da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:42.758944 12553 tablet_service.cc:1467] Processing CreateTablet for tablet d517dc8327d04c85a30e8e86461d19da (DEFAULT_TABLE table=test-table [id=793c68207a5143c6828165acf6ef7dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:42.759009 12356 tablet_service.cc:1467] Processing CreateTablet for tablet d517dc8327d04c85a30e8e86461d19da (DEFAULT_TABLE table=test-table [id=793c68207a5143c6828165acf6ef7dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:42.759239 12553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d517dc8327d04c85a30e8e86461d19da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:42.759320 12356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d517dc8327d04c85a30e8e86461d19da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:42.761639 12664 tablet_bootstrap.cc:492] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: Bootstrap starting.
I20260808 02:16:42.762043 12665 tablet_bootstrap.cc:492] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34: Bootstrap starting.
I20260808 02:16:42.762356 12664 tablet_bootstrap.cc:654] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:42.762495 12666 tablet_bootstrap.cc:492] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: Bootstrap starting.
I20260808 02:16:42.762688 12665 tablet_bootstrap.cc:654] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:42.762974 12664 log.cc:824] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:42.763142 12666 tablet_bootstrap.cc:654] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:42.763211 12665 log.cc:824] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:42.763665 12666 log.cc:824] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:42.763738 12664 tablet_bootstrap.cc:492] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: No bootstrap required, opened a new log
I20260808 02:16:42.763823 12664 ts_tablet_manager.cc:1397] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:42.764057 12665 tablet_bootstrap.cc:492] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34: No bootstrap required, opened a new log
I20260808 02:16:42.764127 12665 ts_tablet_manager.cc:1397] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:42.765673 12664 raft_consensus.cc:348] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10adaec6984153b431636987966f9a" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35505 } } peers { permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 } } peers { permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 } }
I20260808 02:16:42.765872 12664 raft_consensus.cc:374] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:42.765918 12664 raft_consensus.cc:729] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d10adaec6984153b431636987966f9a, State: Initialized, Role: FOLLOWER
I20260808 02:16:42.766037 12664 consensus_queue.cc:260] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10adaec6984153b431636987966f9a" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35505 } } peers { permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 } } peers { permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 } }
I20260808 02:16:42.765983 12665 raft_consensus.cc:348] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10adaec6984153b431636987966f9a" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35505 } } peers { permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 } } peers { permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 } }
I20260808 02:16:42.766125 12665 raft_consensus.cc:374] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:42.766171 12665 raft_consensus.cc:729] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dde7892fe93e409584a25894b720ae34, State: Initialized, Role: FOLLOWER
I20260808 02:16:42.766322 12664 ts_tablet_manager.cc:1428] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 02:16:42.766280 12665 consensus_queue.cc:260] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10adaec6984153b431636987966f9a" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35505 } } peers { permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 } } peers { permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 } }
I20260808 02:16:42.766340 12640 heartbeater.cc:503] Master 127.19.58.254:38015 was elected leader, sending a full tablet report...
I20260808 02:16:42.766553 12665 ts_tablet_manager.cc:1428] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:16:42.766582 12260 heartbeater.cc:503] Master 127.19.58.254:38015 was elected leader, sending a full tablet report...
I20260808 02:16:42.767123 12666 tablet_bootstrap.cc:492] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: No bootstrap required, opened a new log
I20260808 02:16:42.767194 12666 ts_tablet_manager.cc:1397] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 02:16:42.768769 12666 raft_consensus.cc:348] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10adaec6984153b431636987966f9a" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35505 } } peers { permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 } } peers { permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 } }
I20260808 02:16:42.768899 12666 raft_consensus.cc:374] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:42.768923 12666 raft_consensus.cc:729] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 796bc3da5efd492781b4d2deaff3df12, State: Initialized, Role: FOLLOWER
I20260808 02:16:42.769024 12666 consensus_queue.cc:260] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10adaec6984153b431636987966f9a" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35505 } } peers { permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 } } peers { permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 } }
I20260808 02:16:42.769248 12666 ts_tablet_manager.cc:1428] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:42.769280 12450 heartbeater.cc:503] Master 127.19.58.254:38015 was elected leader, sending a full tablet report...
W20260808 02:16:42.820225 12452 tablet.cc:2369] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:42.831491 12670 raft_consensus.cc:482] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:42.831619 12670 raft_consensus.cc:504] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10adaec6984153b431636987966f9a" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35505 } } peers { permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 } } peers { permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 } }
I20260808 02:16:42.831990 12670 leader_election.cc:288] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dde7892fe93e409584a25894b720ae34 (127.19.58.193:38317), 796bc3da5efd492781b4d2deaff3df12 (127.19.58.194:43527)
I20260808 02:16:42.834780 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.832246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:37831 (local address 127.19.58.194:43527)
0808 02:16:42.832309 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.832311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.832461 (+ 150us) server_negotiation.cc:407] Connection header received
0808 02:16:42.832512 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.832513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.832522 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.832570 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:42.832708 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.833186 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.834104 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.834484 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.834544 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.834552 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.834677 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.834711 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 02:16:42.834785 12661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.832250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:34873 (local address 127.19.58.193:38317)
0808 02:16:42.832357 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.832399 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.832407 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:16:42.832432 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.832433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.832442 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.832497 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:42.832663 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.833145 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.834088 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.834469 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.834532 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.834542 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.834679 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.834717 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 02:16:42.834796 12642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.832154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:38317 (local address 127.19.58.195:34873)
0808 02:16:42.832227 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.832238 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:42.832278 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:42.832530 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:42.832531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:42.832533 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:42.832600 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.832602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.833232 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.833233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.834004 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.834006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.834545 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.834546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.834620 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.834623 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:42.834667 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:16:42.834710 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":27}
I20260808 02:16:42.834806 12673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.832163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:43527 (local address 127.19.58.195:37831)
0808 02:16:42.832388 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.832400 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:42.832457 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:42.832580 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:42.832581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:42.832582 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:42.832654 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.832656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.833259 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.833260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.834028 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:42.834030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.834558 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.834559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:42.834618 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.834622 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:42.834666 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:16:42.834706 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":14,"thread_start_us":139,"threads_started":1}
I20260808 02:16:42.835150 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d517dc8327d04c85a30e8e86461d19da" candidate_uuid: "9d10adaec6984153b431636987966f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "796bc3da5efd492781b4d2deaff3df12" is_pre_election: true
I20260808 02:16:42.835175 12199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d517dc8327d04c85a30e8e86461d19da" candidate_uuid: "9d10adaec6984153b431636987966f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dde7892fe93e409584a25894b720ae34" is_pre_election: true
I20260808 02:16:42.835311 12378 raft_consensus.cc:2454] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d10adaec6984153b431636987966f9a in term 0.
I20260808 02:16:42.835317 12199 raft_consensus.cc:2454] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d10adaec6984153b431636987966f9a in term 0.
I20260808 02:16:42.835527 12483 leader_election.cc:302] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 796bc3da5efd492781b4d2deaff3df12, 9d10adaec6984153b431636987966f9a; no voters:
I20260808 02:16:42.835692 12670 raft_consensus.cc:2781] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:16:42.835763 12670 raft_consensus.cc:482] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:16:42.835778 12670 raft_consensus.cc:3037] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:42.836474 12670 raft_consensus.cc:504] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10adaec6984153b431636987966f9a" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35505 } } peers { permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 } } peers { permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 } }
I20260808 02:16:42.836582 12670 leader_election.cc:288] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [CANDIDATE]: Term 1 election: Requested vote from peers dde7892fe93e409584a25894b720ae34 (127.19.58.193:38317), 796bc3da5efd492781b4d2deaff3df12 (127.19.58.194:43527)
I20260808 02:16:42.836742 12378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d517dc8327d04c85a30e8e86461d19da" candidate_uuid: "9d10adaec6984153b431636987966f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "796bc3da5efd492781b4d2deaff3df12"
I20260808 02:16:42.836787 12199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d517dc8327d04c85a30e8e86461d19da" candidate_uuid: "9d10adaec6984153b431636987966f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dde7892fe93e409584a25894b720ae34"
I20260808 02:16:42.836826 12378 raft_consensus.cc:3037] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:42.836853 12199 raft_consensus.cc:3037] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:42.837531 12199 raft_consensus.cc:2454] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d10adaec6984153b431636987966f9a in term 1.
I20260808 02:16:42.837591 12378 raft_consensus.cc:2454] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d10adaec6984153b431636987966f9a in term 1.
I20260808 02:16:42.837731 12481 leader_election.cc:302] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d10adaec6984153b431636987966f9a, dde7892fe93e409584a25894b720ae34; no voters:
I20260808 02:16:42.837821 12670 raft_consensus.cc:2781] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:42.837953 12670 raft_consensus.cc:686] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 1 LEADER]: Becoming Leader. State: Replica: 9d10adaec6984153b431636987966f9a, State: Running, Role: LEADER
I20260808 02:16:42.838030 12670 consensus_queue.cc:237] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10adaec6984153b431636987966f9a" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35505 } } peers { permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 } } peers { permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 } }
I20260808 02:16:42.838765 12002 catalog_manager.cc:5495] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d10adaec6984153b431636987966f9a (127.19.58.195). New cstate: current_term: 1 leader_uuid: "9d10adaec6984153b431636987966f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d10adaec6984153b431636987966f9a" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 35505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 } health_report { overall_health: UNKNOWN } } }
I20260808 02:16:42.851227 12660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.849026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.19.58.195:35505)
0808 02:16:42.849067 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:42.849070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:42.849078 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:16:42.849151 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:42.849152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:42.849169 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:42.849228 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:42.849372 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.849805 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.850623 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:42.850805 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:42.850883 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:42.850985 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:42.851075 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:42.851131 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:42.851144 (+ 13us) server_negotiation.cc:299] Negotiation successful
0808 02:16:42.851160 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:16:42.852833 12378 raft_consensus.cc:1261] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12 [term 1 FOLLOWER]: Refusing update from remote peer 9d10adaec6984153b431636987966f9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:42.852850 12199 raft_consensus.cc:1261] T d517dc8327d04c85a30e8e86461d19da P dde7892fe93e409584a25894b720ae34 [term 1 FOLLOWER]: Refusing update from remote peer 9d10adaec6984153b431636987966f9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:42.853130 12675 consensus_queue.cc:1035] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "796bc3da5efd492781b4d2deaff3df12" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:42.853256 12670 consensus_queue.cc:1035] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dde7892fe93e409584a25894b720ae34" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:42.855562 12680 mvcc.cc:205] Tried to move back new op lower bound from 7316092530082897920 to 7316092530025119744. Current Snapshot: MvccSnapshot[applied={T|T < 7316092530082897920}]
I20260808 02:16:42.857661 12681 mvcc.cc:205] Tried to move back new op lower bound from 7316092530082897920 to 7316092530025119744. Current Snapshot: MvccSnapshot[applied={T|T < 7316092530082897920 or (T in {7316092530082897920})}]
I20260808 02:16:42.857936 12002 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:42.858016 12002 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:42.858650 12002 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35129b963ae4487c89b9569e25f2aba9: Sending DeleteTablet for 3 replicas of tablet d517dc8327d04c85a30e8e86461d19da
I20260808 02:16:42.858963 12553 tablet_service.cc:1514] Processing DeleteTablet for tablet d517dc8327d04c85a30e8e86461d19da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:42 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:51618
I20260808 02:16:42.859217 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 12085
I20260808 02:16:42.859194 12171 tablet_service.cc:1514] Processing DeleteTablet for tablet d517dc8327d04c85a30e8e86461d19da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:42 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:58456
I20260808 02:16:42.864234 12356 tablet_service.cc:1514] Processing DeleteTablet for tablet d517dc8327d04c85a30e8e86461d19da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:42 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:34028
I20260808 02:16:42.866312 12692 tablet_replica.cc:330] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: stopping tablet replica
I20260808 02:16:42.866441 12692 raft_consensus.cc:2229] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:16:42.866549 12692 raft_consensus.cc:2258] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:42.867125 12692 ts_tablet_manager.cc:1905] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:42.869298 12692 ts_tablet_manager.cc:1918] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:42.869369 12692 log.cc:1196] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/d517dc8327d04c85a30e8e86461d19da
I20260808 02:16:42.869632 12692 ts_tablet_manager.cc:1939] T d517dc8327d04c85a30e8e86461d19da P 9d10adaec6984153b431636987966f9a: Deleting consensus metadata
I20260808 02:16:42.870643 11984 catalog_manager.cc:4841] TS 9d10adaec6984153b431636987966f9a (127.19.58.195:35505): tablet d517dc8327d04c85a30e8e86461d19da (table test-table [id=793c68207a5143c6828165acf6ef7dd4]) successfully deleted
I20260808 02:16:42.870785 12693 tablet_replica.cc:330] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: stopping tablet replica
I20260808 02:16:42.870877 12693 raft_consensus.cc:2229] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:42.870959 12693 raft_consensus.cc:2258] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:42.871343 12693 ts_tablet_manager.cc:1905] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:42.872663 12693 ts_tablet_manager.cc:1918] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:42.872728 12693 log.cc:1196] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/d517dc8327d04c85a30e8e86461d19da
I20260808 02:16:42.873016 12693 ts_tablet_manager.cc:1939] T d517dc8327d04c85a30e8e86461d19da P 796bc3da5efd492781b4d2deaff3df12: Deleting consensus metadata
I20260808 02:16:42.873644 11985 catalog_manager.cc:4841] TS 796bc3da5efd492781b4d2deaff3df12 (127.19.58.194:43527): tablet d517dc8327d04c85a30e8e86461d19da (table test-table [id=793c68207a5143c6828165acf6ef7dd4]) successfully deleted
W20260808 02:16:42.876269 11983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:38317 (error 108)
I20260808 02:16:42.876518 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 12267
I20260808 02:16:42.877946 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:42.876599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:38317 (local address 127.0.0.1:58468)
0808 02:16:42.877870 (+ 1271us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:42.877911 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:38317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1250}
W20260808 02:16:42.883106 11983 catalog_manager.cc:4584] TS dde7892fe93e409584a25894b720ae34 (127.19.58.193:38317): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d517dc8327d04c85a30e8e86461d19da: Network error: Client connection negotiation failed: client connection to 127.19.58.193:38317: connect: Connection refused (error 111)
I20260808 02:16:42.883319 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 12461
I20260808 02:16:42.893227 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 11959
2026-08-08T02:16:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2026 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T02:16:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:42Z Disabled control of system clock
I20260808 02:16:42.924082 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:43265
--webserver_interface=127.19.58.254
--webserver_port=0
--builtin_ntp_servers=127.19.58.212:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:43265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:43.004705 12703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:43.004913 12703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:43.004933 12703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:43.006075 12703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:43.006116 12703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:43.006129 12703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:43.006139 12703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:43.006150 12703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:43.007495 12703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:43265
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:43265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:43.007745 12703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:43.008029 12703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:43.012490 12716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:43.012509 12712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:43.012547 12711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:43.012840 12703 server_base.cc:1034] running on GCE node
I20260808 02:16:43.013062 12703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:43.013293 12703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:43.014425 12703 hybrid_clock.cc:648] HybridClock initialized: now 1786155403014414 us; error 30 us; skew 500 ppm
I20260808 02:16:43.015646 12703 webserver.cc:460] Webserver started at http://127.19.58.254:38803/ using document root <none> and password file <none>
I20260808 02:16:43.015887 12703 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:43.015938 12703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:43.016068 12703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:43.017045 12703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "1b3e2d419432488280e36c05bc156a2b"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "876aae212e4c62f6c5fde0dc14d42bea"
server_key_iv: "42d4276e219d00399b3a13a7a37b7de3"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.017364 12703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "1b3e2d419432488280e36c05bc156a2b"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "876aae212e4c62f6c5fde0dc14d42bea"
server_key_iv: "42d4276e219d00399b3a13a7a37b7de3"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.018571 12703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:16:43.019187 12728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:43.019320 12703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:43.019408 12703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "1b3e2d419432488280e36c05bc156a2b"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "876aae212e4c62f6c5fde0dc14d42bea"
server_key_iv: "42d4276e219d00399b3a13a7a37b7de3"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.019466 12703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:43.039098 12703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:43.039431 12703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:43.039566 12703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:43.043365 12703 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:43265
I20260808 02:16:43.043409 12806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:43265 every 8 connection(s)
I20260808 02:16:43.043763 12703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:43.044688 12807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:43.047233 12807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b: Bootstrap starting.
I20260808 02:16:43.047857 12807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:43.048173 12807 log.cc:824] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:43.048837 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 12703
I20260808 02:16:43.048867 12807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b: No bootstrap required, opened a new log
I20260808 02:16:43.048949 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:43.049121 19691 external_mini_cluster.cc:1428] Setting key ad40840b046648dcefd7caf63efe01c0
I20260808 02:16:43.050887 12807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3e2d419432488280e36c05bc156a2b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43265 } }
I20260808 02:16:43.051056 12807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:43.051105 12807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b3e2d419432488280e36c05bc156a2b, State: Initialized, Role: FOLLOWER
I20260808 02:16:43.051221 12807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3e2d419432488280e36c05bc156a2b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43265 } }
I20260808 02:16:43.051293 12807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:43.051324 12807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:43.051380 12807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:43.052116 12807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3e2d419432488280e36c05bc156a2b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43265 } }
I20260808 02:16:43.052225 12807 leader_election.cc:302] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b3e2d419432488280e36c05bc156a2b; no voters:
I20260808 02:16:43.052389 12807 leader_election.cc:288] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:43.052577 12815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:43.052639 12807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:43.053028 12815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [term 1 LEADER]: Becoming Leader. State: Replica: 1b3e2d419432488280e36c05bc156a2b, State: Running, Role: LEADER
I20260808 02:16:43.053526 12815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3e2d419432488280e36c05bc156a2b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43265 } }
I20260808 02:16:43.055363 12816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b3e2d419432488280e36c05bc156a2b. Latest consensus state: current_term: 1 leader_uuid: "1b3e2d419432488280e36c05bc156a2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3e2d419432488280e36c05bc156a2b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43265 } } }
I20260808 02:16:43.055686 12816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:43.055544 12814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b3e2d419432488280e36c05bc156a2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3e2d419432488280e36c05bc156a2b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 43265 } } }
I20260808 02:16:43.055872 12814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:43.056263 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.049596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46482 (local address 127.19.58.254:43265)
0808 02:16:43.049801 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.049810 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.049842 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:16:43.050856 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.050873 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.050951 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.051106 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:43.051322 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.053039 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.053920 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.054516 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.054630 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.055424 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:43.055428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:43.055428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:43.055616 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:43.055661 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:43.055736 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:43.055849 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.055986 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.056066 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
W20260808 02:16:43.056311 12830 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:16:43.056347 12830 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:16:43.056517 12833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:43.056738 12833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:43.058063 12833 catalog_manager.cc:1347] Generated new cluster ID: bb699f0caaed4cb68c7f0b01502f27d1
I20260808 02:16:43.058117 12833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:43.070540 12833 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:43.071203 12833 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:43.072953 12833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b: Generated new TSK 0
I20260808 02:16:43.073122 12833 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:43.080451 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43265
--builtin_ntp_servers=127.19.58.212:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:43.162184 12836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:43.162393 12836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:43.162423 12836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:43.163517 12836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:43.163556 12836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:43.163589 12836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:16:43.164919 12836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:43.165136 12836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:43.165431 12836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:43.170284 12844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:43.170360 12836 server_base.cc:1034] running on GCE node
W20260808 02:16:43.170287 12848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:43.170531 12845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:43.170776 12836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:43.171011 12836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:43.172147 12836 hybrid_clock.cc:648] HybridClock initialized: now 1786155403172134 us; error 32 us; skew 500 ppm
I20260808 02:16:43.173291 12836 webserver.cc:460] Webserver started at http://127.19.58.193:33943/ using document root <none> and password file <none>
I20260808 02:16:43.173501 12836 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:43.173552 12836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:43.173681 12836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:43.174681 12836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "0bacccd792e1426e887daf9ca5640170"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "f55a2bd9e4f97bacf1de88b49dbc58eb"
server_key_iv: "8eb19c96e203edc3106a67d17eb38ec3"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.175020 12836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "0bacccd792e1426e887daf9ca5640170"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "f55a2bd9e4f97bacf1de88b49dbc58eb"
server_key_iv: "8eb19c96e203edc3106a67d17eb38ec3"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.176430 12836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:16:43.177114 12855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:43.177255 12836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:43.177336 12836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "0bacccd792e1426e887daf9ca5640170"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "f55a2bd9e4f97bacf1de88b49dbc58eb"
server_key_iv: "8eb19c96e203edc3106a67d17eb38ec3"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.177393 12836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:43.189482 12836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:43.189793 12836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:43.189926 12836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:43.190167 12836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:43.190495 12836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:43.190528 12836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:43.190557 12836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:43.190574 12836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:43.196372 12836 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:38869
I20260808 02:16:43.196533 13024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:38869 every 8 connection(s)
I20260808 02:16:43.196738 12836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:43.201792 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.197451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:42251 (local address 127.19.58.254:43265)
0808 02:16:43.197506 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.197508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.197770 (+ 262us) server_negotiation.cc:407] Connection header received
0808 02:16:43.198262 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.198264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.198278 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.198328 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:43.198983 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.199501 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.200502 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.200703 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.200760 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.201534 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:43.201535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:43.201535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:43.201540 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:43.201554 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:43.201558 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:43.201604 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.201680 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.201711 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:16:43.201974 13027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.197441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:43265 (local address 127.19.58.193:42251)
0808 02:16:43.197646 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.197681 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:43.198166 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:43.198410 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:43.198414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:43.198422 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:43.198895 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.198900 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.199591 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.199592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.200415 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.200418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.200775 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.200776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.200898 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.201400 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:43.201401 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:43.201430 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:43.201451 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:43.201610 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:43.201615 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:43.201618 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:43.201678 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 02:16:43.201781 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":99,"threads_started":1}
I20260808 02:16:43.202448 13025 heartbeater.cc:347] Connected to a master server at 127.19.58.254:43265
I20260808 02:16:43.202538 13025 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:43.202708 13025 heartbeater.cc:511] Master 127.19.58.254:43265 requested a full tablet report, sending...
I20260808 02:16:43.203198 12755 ts_manager.cc:194] Registered new tserver with Master: 0bacccd792e1426e887daf9ca5640170 (127.19.58.193:38869)
I20260808 02:16:43.203879 12755 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.193:42251
I20260808 02:16:43.205992 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 12836
I20260808 02:16:43.206077 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:16:43.206261 19691 external_mini_cluster.cc:1428] Setting key df7001f3ced35186dbf4a29eb79672c1
I20260808 02:16:43.207116 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43265
--builtin_ntp_servers=127.19.58.212:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:43.287745 13028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:43.287952 13028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:43.287981 13028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:43.289083 13028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:43.289119 13028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:43.289150 13028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:16:43.290485 13028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:43.290699 13028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:43.290970 13028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:43.295668 13041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:43.295852 13038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:43.295933 13028 server_base.cc:1034] running on GCE node
W20260808 02:16:43.295738 13036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:43.296248 13028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:43.296473 13028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:43.297609 13028 hybrid_clock.cc:648] HybridClock initialized: now 1786155403297595 us; error 29 us; skew 500 ppm
I20260808 02:16:43.298766 13028 webserver.cc:460] Webserver started at http://127.19.58.194:36017/ using document root <none> and password file <none>
I20260808 02:16:43.298972 13028 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:43.299022 13028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:43.299140 13028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:43.300156 13028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "622e9192d493432f9533f3096fdcff55"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "f66730d13b1701f8f9bd88e6868b6acb"
server_key_iv: "6fa6db3776684b0f4ccce85228f3dd39"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.300478 13028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "622e9192d493432f9533f3096fdcff55"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "f66730d13b1701f8f9bd88e6868b6acb"
server_key_iv: "6fa6db3776684b0f4ccce85228f3dd39"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.301671 13028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:16:43.302402 13051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:43.302572 13028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:43.302651 13028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "622e9192d493432f9533f3096fdcff55"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "f66730d13b1701f8f9bd88e6868b6acb"
server_key_iv: "6fa6db3776684b0f4ccce85228f3dd39"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.302708 13028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:43.331804 13028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:43.332091 13028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:43.332223 13028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:43.332443 13028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:43.332793 13028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:43.332831 13028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:43.332859 13028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:43.332876 13028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:43.338088 13028 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:38371
I20260808 02:16:43.338155 13207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:38371 every 8 connection(s)
I20260808 02:16:43.338449 13028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:16:43.342356 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 13028
I20260808 02:16:43.342463 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:16:43.342638 19691 external_mini_cluster.cc:1428] Setting key dc4d1afb113d2bd2d397a2ccaca140e1
I20260808 02:16:43.343411 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.339095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:47377 (local address 127.19.58.254:43265)
0808 02:16:43.339138 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.339140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.339383 (+ 243us) server_negotiation.cc:407] Connection header received
0808 02:16:43.339926 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.339927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.339939 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.339993 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:43.340639 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.341126 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.342050 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.342240 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.342296 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.343112 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:43.343113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:43.343113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:43.343117 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:43.343131 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:43.343135 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:43.343199 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.343274 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.343302 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:16:43.343564 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.339079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:43265 (local address 127.19.58.194:47377)
0808 02:16:43.339268 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.339301 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:43.339829 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:43.340059 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:43.340063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:43.340072 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:43.340552 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.340556 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.341210 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.341211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.341970 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.341972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.342313 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.342313 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.342439 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.342982 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:43.342982 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:43.343018 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:43.343043 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:43.343205 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:43.343210 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:43.343213 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:43.343278 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 02:16:43.343379 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260808 02:16:43.343601 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43265
--builtin_ntp_servers=127.19.58.212:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:16:43.344036 13208 heartbeater.cc:347] Connected to a master server at 127.19.58.254:43265
I20260808 02:16:43.344126 13208 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:43.344311 13208 heartbeater.cc:511] Master 127.19.58.254:43265 requested a full tablet report, sending...
I20260808 02:16:43.344650 12755 ts_manager.cc:194] Registered new tserver with Master: 622e9192d493432f9533f3096fdcff55 (127.19.58.194:38371)
I20260808 02:16:43.345048 12755 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.194:47377
W20260808 02:16:43.425920 13212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:43.426118 13212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:43.426139 13212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:43.427350 13212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:43.427392 13212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:43.427416 13212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:16:43.428788 13212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:43265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:43.429044 13212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:43.429319 13212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:43.434041 13229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:43.434055 13227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:43.434093 13224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:43.434250 13212 server_base.cc:1034] running on GCE node
I20260808 02:16:43.434451 13212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:43.434672 13212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:43.435809 13212 hybrid_clock.cc:648] HybridClock initialized: now 1786155403435792 us; error 35 us; skew 500 ppm
I20260808 02:16:43.437057 13212 webserver.cc:460] Webserver started at http://127.19.58.195:46171/ using document root <none> and password file <none>
I20260808 02:16:43.437288 13212 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:43.437337 13212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:43.437565 13212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:43.438721 13212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "5a3a9c57ca7a4369af113810acddc355"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "2cd1fb6f016bc7bd0fa17524f884b3f7"
server_key_iv: "b789e92a332299f5543fe74ea06deab1"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.439070 13212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "5a3a9c57ca7a4369af113810acddc355"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "2cd1fb6f016bc7bd0fa17524f884b3f7"
server_key_iv: "b789e92a332299f5543fe74ea06deab1"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.440394 13212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:16:43.441044 13243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:43.441183 13212 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:43.441263 13212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "5a3a9c57ca7a4369af113810acddc355"
format_stamp: "Formatted at 2026-08-08 02:16:43 on dist-test-slave-1l3l"
server_key: "2cd1fb6f016bc7bd0fa17524f884b3f7"
server_key_iv: "b789e92a332299f5543fe74ea06deab1"
server_key_version: "encryptionkey@0"
I20260808 02:16:43.441331 13212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:43.448613 13212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:43.448848 13212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:43.448962 13212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:43.449146 13212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:43.449457 13212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:43.449492 13212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:43.449519 13212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:43.449542 13212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:43.455233 13212 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:41461
I20260808 02:16:43.455287 13398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:41461 every 8 connection(s)
I20260808 02:16:43.455601 13212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:16:43.459003 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 13212
I20260808 02:16:43.459108 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:16:43.459288 19691 external_mini_cluster.cc:1428] Setting key 06fbd1452b41ed97258b5f0ed2ae99dd
I20260808 02:16:43.460419 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.456228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:57243 (local address 127.19.58.254:43265)
0808 02:16:43.456303 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.456305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.456503 (+ 198us) server_negotiation.cc:407] Connection header received
0808 02:16:43.456920 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.456922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.456931 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.456979 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:43.457572 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.458059 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.459015 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.459300 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.459363 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.460200 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:43.460201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:43.460201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:43.460205 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:43.460221 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:43.460224 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:43.460263 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.460338 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.460361 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 02:16:43.460604 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.456209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:43265 (local address 127.19.58.195:57243)
0808 02:16:43.456392 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.456422 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:43.456847 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:43.457027 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:43.457030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:43.457037 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:43.457498 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.457502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.458130 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.458131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.458936 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.458938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.459393 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.459394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.459545 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.460082 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:43.460083 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:43.460110 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:43.460130 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:43.460274 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:43.460278 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:43.460280 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:16:43.460338 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 02:16:43.460412 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260808 02:16:43.461024 13403 heartbeater.cc:347] Connected to a master server at 127.19.58.254:43265
I20260808 02:16:43.461105 13403 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:43.461239 13403 heartbeater.cc:511] Master 127.19.58.254:43265 requested a full tablet report, sending...
I20260808 02:16:43.461560 12755 ts_manager.cc:194] Registered new tserver with Master: 5a3a9c57ca7a4369af113810acddc355 (127.19.58.195:41461)
I20260808 02:16:43.461920 12755 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.195:57243
I20260808 02:16:43.471043 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:16:43.474954 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.472392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.19.58.254:43265)
0808 02:16:43.472461 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.472463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.472532 (+ 69us) server_negotiation.cc:407] Connection header received
0808 02:16:43.472581 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.472582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.472593 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.472628 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:43.472755 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.473220 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.473950 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.474125 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.474169 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.474779 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:43.474780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:43.474780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:43.474784 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:43.474793 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:43.474795 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:43.474827 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.474870 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.474889 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:16:43.476490 12755 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:16:43.477928 12755 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:16:43.484289 13421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.480234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54616 (local address 127.19.58.195:41461)
0808 02:16:43.480418 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.480423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.480434 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:16:43.480473 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.480475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.480490 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.480567 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:43.480774 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.481902 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.483224 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.483971 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.484035 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.484125 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.484199 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.484227 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260808 02:16:43.484314 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.479953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:41461 (local address 127.0.0.1:54616)
0808 02:16:43.480276 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.480287 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:43.480338 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:43.480595 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:43.480598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:43.480600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:43.480685 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.480687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.481983 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.481984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.483130 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.483133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.484041 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.484041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.484115 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.484129 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 02:16:43.484189 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 02:16:43.484238 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":24,"thread_start_us":108,"threads_started":1}
I20260808 02:16:43.484925 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.480155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:38869 (local address 127.0.0.1:40112)
0808 02:16:43.480656 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.480666 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:43.480715 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:43.480965 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:43.480966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:43.480967 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:43.481009 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.481011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.481859 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.481860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.483130 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.483133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.484698 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.484699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.484754 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.484758 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:43.484804 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:16:43.484843 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":12,"thread_start_us":64,"threads_started":1}
I20260808 02:16:43.485069 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.479927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:38371 (local address 127.0.0.1:48474)
0808 02:16:43.480115 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.480197 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:43.480262 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:43.480607 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:43.480608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:43.480610 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:43.480685 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.480687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.483224 (+ 2537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.483224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.483930 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.483932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.484823 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.484824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.484892 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.484911 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 02:16:43.484964 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 02:16:43.485002 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":3,"thread_start_us":55,"threads_started":1}
I20260808 02:16:43.485110 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.480026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48474 (local address 127.19.58.194:38371)
0808 02:16:43.480447 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.480450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.480459 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:16:43.480493 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.480495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.480508 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.480606 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:43.480775 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.481909 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.484008 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.484727 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.484791 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.484871 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.484993 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.485031 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":107,"threads_started":1}
I20260808 02:16:43.485189 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.480354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.19.58.193:38869)
0808 02:16:43.480529 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.480533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.480776 (+ 243us) server_negotiation.cc:407] Connection header received
0808 02:16:43.480858 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.480861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.480895 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.480969 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:43.481062 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.481784 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.483226 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.483976 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.484040 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.484125 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.485067 (+ 942us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.485112 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260808 02:16:43.485111 13304 tablet_service.cc:1467] Processing CreateTablet for tablet 93151c8526e64ee1aa60187ed01c08a4 (DEFAULT_TABLE table=test-table [id=8affc94d59514f39be85085175c81f23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:43.485518 13304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93151c8526e64ee1aa60187ed01c08a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:43.485786 13118 tablet_service.cc:1467] Processing CreateTablet for tablet 93151c8526e64ee1aa60187ed01c08a4 (DEFAULT_TABLE table=test-table [id=8affc94d59514f39be85085175c81f23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:43.486143 13118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93151c8526e64ee1aa60187ed01c08a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:43.486053 12928 tablet_service.cc:1467] Processing CreateTablet for tablet 93151c8526e64ee1aa60187ed01c08a4 (DEFAULT_TABLE table=test-table [id=8affc94d59514f39be85085175c81f23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:43.486353 12928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93151c8526e64ee1aa60187ed01c08a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:43.488010 13424 tablet_bootstrap.cc:492] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: Bootstrap starting.
I20260808 02:16:43.488683 13424 tablet_bootstrap.cc:654] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:43.488993 13424 log.cc:824] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:43.489266 13425 tablet_bootstrap.cc:492] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: Bootstrap starting.
I20260808 02:16:43.489722 13424 tablet_bootstrap.cc:492] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: No bootstrap required, opened a new log
I20260808 02:16:43.489780 13424 ts_tablet_manager.cc:1397] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:43.489862 13425 tablet_bootstrap.cc:654] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:43.490115 13425 log.cc:824] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:43.490406 13429 tablet_bootstrap.cc:492] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170: Bootstrap starting.
I20260808 02:16:43.490890 13425 tablet_bootstrap.cc:492] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: No bootstrap required, opened a new log
I20260808 02:16:43.491055 13425 ts_tablet_manager.cc:1397] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:43.491187 13429 tablet_bootstrap.cc:654] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:43.491532 13429 log.cc:824] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:43.492013 13424 raft_consensus.cc:348] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 } } peers { permanent_uuid: "5a3a9c57ca7a4369af113810acddc355" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41461 } } peers { permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 } }
I20260808 02:16:43.492149 13424 raft_consensus.cc:374] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:43.492182 13424 raft_consensus.cc:729] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3a9c57ca7a4369af113810acddc355, State: Initialized, Role: FOLLOWER
I20260808 02:16:43.492288 13424 consensus_queue.cc:260] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 } } peers { permanent_uuid: "5a3a9c57ca7a4369af113810acddc355" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41461 } } peers { permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 } }
I20260808 02:16:43.492532 13424 ts_tablet_manager.cc:1428] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 02:16:43.492548 13429 tablet_bootstrap.cc:492] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170: No bootstrap required, opened a new log
I20260808 02:16:43.492607 13429 ts_tablet_manager.cc:1397] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 02:16:43.492637 13403 heartbeater.cc:503] Master 127.19.58.254:43265 was elected leader, sending a full tablet report...
I20260808 02:16:43.493468 13425 raft_consensus.cc:348] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 } } peers { permanent_uuid: "5a3a9c57ca7a4369af113810acddc355" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41461 } } peers { permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 } }
I20260808 02:16:43.493623 13425 raft_consensus.cc:374] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:43.493669 13425 raft_consensus.cc:729] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622e9192d493432f9533f3096fdcff55, State: Initialized, Role: FOLLOWER
I20260808 02:16:43.493777 13425 consensus_queue.cc:260] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 } } peers { permanent_uuid: "5a3a9c57ca7a4369af113810acddc355" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41461 } } peers { permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 } }
I20260808 02:16:43.494029 13425 ts_tablet_manager.cc:1428] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:43.494066 13208 heartbeater.cc:503] Master 127.19.58.254:43265 was elected leader, sending a full tablet report...
I20260808 02:16:43.495375 13429 raft_consensus.cc:348] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 } } peers { permanent_uuid: "5a3a9c57ca7a4369af113810acddc355" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41461 } } peers { permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 } }
I20260808 02:16:43.495497 13429 raft_consensus.cc:374] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:43.495517 13429 raft_consensus.cc:729] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bacccd792e1426e887daf9ca5640170, State: Initialized, Role: FOLLOWER
I20260808 02:16:43.495595 13429 consensus_queue.cc:260] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 } } peers { permanent_uuid: "5a3a9c57ca7a4369af113810acddc355" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41461 } } peers { permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 } }
I20260808 02:16:43.495838 13429 ts_tablet_manager.cc:1428] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 02:16:43.495898 13025 heartbeater.cc:503] Master 127.19.58.254:43265 was elected leader, sending a full tablet report...
W20260808 02:16:43.588994 13209 tablet.cc:2369] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:16:43.697423 13026 tablet.cc:2369] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:16:43.706133 13404 tablet.cc:2369] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:43.726483 13435 raft_consensus.cc:482] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:43.726624 13435 raft_consensus.cc:504] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 } } peers { permanent_uuid: "5a3a9c57ca7a4369af113810acddc355" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41461 } } peers { permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 } }
I20260808 02:16:43.727000 13435 leader_election.cc:288] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 622e9192d493432f9533f3096fdcff55 (127.19.58.194:38371), 0bacccd792e1426e887daf9ca5640170 (127.19.58.193:38869)
I20260808 02:16:43.729811 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.727207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:38869 (local address 127.19.58.195:45425)
0808 02:16:43.727258 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.727270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:43.727316 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:43.727452 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:43.727453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:43.727455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:43.727516 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.727518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.728265 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.728266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.729039 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.729041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.729590 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.729591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.729657 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.729661 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:43.729706 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 02:16:43.729740 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 02:16:43.729832 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.727279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:45425 (local address 127.19.58.193:38869)
0808 02:16:43.727325 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.727327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.727337 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:16:43.727367 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.727368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.727380 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.727438 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:43.727596 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.728191 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.729133 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.729514 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.729567 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.729574 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.729726 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.729761 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 02:16:43.729895 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.727289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:36621 (local address 127.19.58.194:38371)
0808 02:16:43.727338 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.727340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.727549 (+ 209us) server_negotiation.cc:407] Connection header received
0808 02:16:43.727582 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.727583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.727593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.727640 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:43.727813 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.728292 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.729204 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.729564 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.729632 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.729639 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.729792 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.729837 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 02:16:43.729921 13440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.727212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:38371 (local address 127.19.58.195:36621)
0808 02:16:43.727475 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.727484 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:43.727530 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:43.727656 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:43.727657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:43.727658 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:43.727750 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.727752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.728363 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.728364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.729117 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:43.729120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.729644 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.729644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:43.729716 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.729720 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:43.729773 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 02:16:43.729813 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":53,"thread_start_us":104,"threads_started":1}
I20260808 02:16:43.730280 13144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93151c8526e64ee1aa60187ed01c08a4" candidate_uuid: "5a3a9c57ca7a4369af113810acddc355" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622e9192d493432f9533f3096fdcff55" is_pre_election: true
I20260808 02:16:43.730343 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93151c8526e64ee1aa60187ed01c08a4" candidate_uuid: "5a3a9c57ca7a4369af113810acddc355" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bacccd792e1426e887daf9ca5640170" is_pre_election: true
I20260808 02:16:43.730432 13144 raft_consensus.cc:2454] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a3a9c57ca7a4369af113810acddc355 in term 0.
I20260808 02:16:43.730487 12956 raft_consensus.cc:2454] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a3a9c57ca7a4369af113810acddc355 in term 0.
I20260808 02:16:43.730683 13249 leader_election.cc:302] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3a9c57ca7a4369af113810acddc355, 622e9192d493432f9533f3096fdcff55; no voters:
I20260808 02:16:43.730815 13435 raft_consensus.cc:2781] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:16:43.730886 13435 raft_consensus.cc:482] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:16:43.730911 13435 raft_consensus.cc:3037] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:43.731621 13435 raft_consensus.cc:504] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 } } peers { permanent_uuid: "5a3a9c57ca7a4369af113810acddc355" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41461 } } peers { permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 } }
I20260808 02:16:43.731772 13435 leader_election.cc:288] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [CANDIDATE]: Term 1 election: Requested vote from peers 622e9192d493432f9533f3096fdcff55 (127.19.58.194:38371), 0bacccd792e1426e887daf9ca5640170 (127.19.58.193:38869)
I20260808 02:16:43.731920 13144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93151c8526e64ee1aa60187ed01c08a4" candidate_uuid: "5a3a9c57ca7a4369af113810acddc355" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622e9192d493432f9533f3096fdcff55"
I20260808 02:16:43.731922 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93151c8526e64ee1aa60187ed01c08a4" candidate_uuid: "5a3a9c57ca7a4369af113810acddc355" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bacccd792e1426e887daf9ca5640170"
I20260808 02:16:43.731995 12956 raft_consensus.cc:3037] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:43.731995 13144 raft_consensus.cc:3037] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:43.732883 13144 raft_consensus.cc:2454] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a3a9c57ca7a4369af113810acddc355 in term 1.
I20260808 02:16:43.732883 12956 raft_consensus.cc:2454] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a3a9c57ca7a4369af113810acddc355 in term 1.
I20260808 02:16:43.733078 13246 leader_election.cc:302] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bacccd792e1426e887daf9ca5640170, 5a3a9c57ca7a4369af113810acddc355; no voters:
I20260808 02:16:43.733168 13435 raft_consensus.cc:2781] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:43.733314 13435 raft_consensus.cc:686] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 1 LEADER]: Becoming Leader. State: Replica: 5a3a9c57ca7a4369af113810acddc355, State: Running, Role: LEADER
I20260808 02:16:43.733418 13435 consensus_queue.cc:237] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 } } peers { permanent_uuid: "5a3a9c57ca7a4369af113810acddc355" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41461 } } peers { permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 } }
I20260808 02:16:43.734193 12754 catalog_manager.cc:5495] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a3a9c57ca7a4369af113810acddc355 (127.19.58.195). New cstate: current_term: 1 leader_uuid: "5a3a9c57ca7a4369af113810acddc355" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3a9c57ca7a4369af113810acddc355" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 41461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 } health_report { overall_health: UNKNOWN } } }
I20260808 02:16:43.768096 13421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.765937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54630 (local address 127.19.58.195:41461)
0808 02:16:43.766007 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:43.766009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:43.766016 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:16:43.766056 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:43.766057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:43.766077 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:43.766136 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:43.766258 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.766740 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.767434 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:43.767629 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:43.767683 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:43.767812 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:43.767926 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:43.767983 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:43.767998 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 02:16:43.768014 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:16:43.769889 13144 raft_consensus.cc:1261] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55 [term 1 FOLLOWER]: Refusing update from remote peer 5a3a9c57ca7a4369af113810acddc355: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:43.769878 12956 raft_consensus.cc:1261] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170 [term 1 FOLLOWER]: Refusing update from remote peer 5a3a9c57ca7a4369af113810acddc355: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:43.770252 13441 consensus_queue.cc:1035] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bacccd792e1426e887daf9ca5640170" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 38869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:43.770393 13435 consensus_queue.cc:1035] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [LEADER]: Connected to new peer: Peer: permanent_uuid: "622e9192d493432f9533f3096fdcff55" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 38371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:43.773366 13445 mvcc.cc:205] Tried to move back new op lower bound from 7316092533838770176 to 7316092533692526592. Current Snapshot: MvccSnapshot[applied={T|T < 7316092533838770176}]
I20260808 02:16:43.775271 12754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:43.775403 12754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:43.776132 12754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b3e2d419432488280e36c05bc156a2b: Sending DeleteTablet for 3 replicas of tablet 93151c8526e64ee1aa60187ed01c08a4
I20260808 02:16:43.776455 13118 tablet_service.cc:1514] Processing DeleteTablet for tablet 93151c8526e64ee1aa60187ed01c08a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:43 UTC) from {username='slave'} at 127.0.0.1:48474
I20260808 02:16:43.776471 13304 tablet_service.cc:1514] Processing DeleteTablet for tablet 93151c8526e64ee1aa60187ed01c08a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:43 UTC) from {username='slave'} at 127.0.0.1:54616
I20260808 02:16:43.776523 12928 tablet_service.cc:1514] Processing DeleteTablet for tablet 93151c8526e64ee1aa60187ed01c08a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:43 UTC) from {username='slave'} at 127.0.0.1:40112
I20260808 02:16:43.776736 13463 tablet_replica.cc:330] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170: stopping tablet replica
I20260808 02:16:43.776736 13460 tablet_replica.cc:330] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: stopping tablet replica
I20260808 02:16:43.776845 13463 raft_consensus.cc:2229] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:43.776913 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 12836
I20260808 02:16:43.776973 13463 raft_consensus.cc:2258] T 93151c8526e64ee1aa60187ed01c08a4 P 0bacccd792e1426e887daf9ca5640170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:43.776889 13462 tablet_replica.cc:330] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: stopping tablet replica
I20260808 02:16:43.777146 13460 raft_consensus.cc:2229] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:43.777369 13462 raft_consensus.cc:2229] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:16:43.777508 13460 raft_consensus.cc:2258] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:43.777675 13462 raft_consensus.cc:2258] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:43.779891 13460 ts_tablet_manager.cc:1905] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:43.781271 13460 ts_tablet_manager.cc:1918] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:43.781337 13460 log.cc:1196] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/93151c8526e64ee1aa60187ed01c08a4
I20260808 02:16:43.781567 13460 ts_tablet_manager.cc:1939] T 93151c8526e64ee1aa60187ed01c08a4 P 622e9192d493432f9533f3096fdcff55: Deleting consensus metadata
I20260808 02:16:43.781919 12735 catalog_manager.cc:4841] TS 622e9192d493432f9533f3096fdcff55 (127.19.58.194:38371): tablet 93151c8526e64ee1aa60187ed01c08a4 (table test-table [id=8affc94d59514f39be85085175c81f23]) successfully deleted
I20260808 02:16:43.783994 13462 ts_tablet_manager.cc:1905] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:43.785225 13462 ts_tablet_manager.cc:1918] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:43.785290 13462 log.cc:1196] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/93151c8526e64ee1aa60187ed01c08a4
I20260808 02:16:43.785554 13462 ts_tablet_manager.cc:1939] T 93151c8526e64ee1aa60187ed01c08a4 P 5a3a9c57ca7a4369af113810acddc355: Deleting consensus metadata
I20260808 02:16:43.785894 12733 catalog_manager.cc:4841] TS 5a3a9c57ca7a4369af113810acddc355 (127.19.58.195:41461): tablet 93151c8526e64ee1aa60187ed01c08a4 (table test-table [id=8affc94d59514f39be85085175c81f23]) successfully deleted
W20260808 02:16:43.786849 12733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:38869 (error 108)
I20260808 02:16:43.787211 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 13028
I20260808 02:16:43.787312 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:43.787177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:38869 (local address 127.0.0.1:40114)
0808 02:16:43.787225 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:43.787274 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:38869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 02:16:43.805703 12733 catalog_manager.cc:4584] TS 0bacccd792e1426e887daf9ca5640170 (127.19.58.193:38869): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93151c8526e64ee1aa60187ed01c08a4: Network error: Client connection negotiation failed: client connection to 127.19.58.193:38869: connect: Connection refused (error 111)
I20260808 02:16:43.810138 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 13212
I20260808 02:16:43.819041 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 12703
2026-08-08T02:16:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (925 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:43 dist-test-slave-1l3l krb5kdc[13467](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:43 dist-test-slave-1l3l krb5kdc[13467](info): set up 2 sockets
Aug 08 02:16:43 dist-test-slave-1l3l krb5kdc[13467](info): commencing operation
krb5kdc: starting...
W20260808 02:16:44.943903 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:16:45 dist-test-slave-1l3l krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:16:45.062506 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40839
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:40839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:45.142519 13489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:45.142710 13489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:45.142731 13489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:45.143899 13489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:45.143941 13489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:45.143953 13489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:45.143966 13489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:45.143990 13489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:45.145432 13489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:40839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:45.145646 13489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:45.145849 13489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:45.150595 13495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:45.150604 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:45.150604 13497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:45.150682 13489 server_base.cc:1034] running on GCE node
I20260808 02:16:45.151067 13489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:45.151333 13489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:45.152465 13489 hybrid_clock.cc:648] HybridClock initialized: now 1786155405152453 us; error 32 us; skew 500 ppm
Aug 08 02:16:45 dist-test-slave-1l3l krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155405, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:45.154886 13489 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:16:45.155323 13489 webserver.cc:460] Webserver started at http://127.19.58.254:37783/ using document root <none> and password file <none>
I20260808 02:16:45.155532 13489 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:45.155584 13489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:45.155736 13489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:45.156821 13489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "fc482211cb2b4a05bfcc386610cf864d"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "6082e0b4db18188638d62b63ae282b71"
server_key_iv: "ffc0c04433bfcbca276b2bcb6e27858e"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.157168 13489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "fc482211cb2b4a05bfcc386610cf864d"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "6082e0b4db18188638d62b63ae282b71"
server_key_iv: "ffc0c04433bfcbca276b2bcb6e27858e"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.158437 13489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:16:45.159127 13513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.159286 13489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:16:45.159369 13489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "fc482211cb2b4a05bfcc386610cf864d"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "6082e0b4db18188638d62b63ae282b71"
server_key_iv: "ffc0c04433bfcbca276b2bcb6e27858e"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.159437 13489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:45.169929 13489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:45.171959 13489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:45.172086 13489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:45.175770 13489 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:40839
I20260808 02:16:45.175822 13584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:40839 every 8 connection(s)
I20260808 02:16:45.176105 13489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:45.176831 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 13489
I20260808 02:16:45.176923 13585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:45.176939 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:45.177117 19691 external_mini_cluster.cc:1428] Setting key 4aa8ca9ef13232ac12fc01498402015b
I20260808 02:16:45.179569 13585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d: Bootstrap starting.
I20260808 02:16:45.180167 13585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:45.180423 13585 log.cc:824] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:45.181048 13585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d: No bootstrap required, opened a new log
I20260808 02:16:45.182935 13585 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc482211cb2b4a05bfcc386610cf864d" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40839 } }
Aug 08 02:16:45 dist-test-slave-1l3l krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:45.183099 13585 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:45.183135 13585 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc482211cb2b4a05bfcc386610cf864d, State: Initialized, Role: FOLLOWER
I20260808 02:16:45.183259 13585 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc482211cb2b4a05bfcc386610cf864d" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40839 } }
I20260808 02:16:45.183338 13585 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:45.183360 13585 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:45.183398 13585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:45.183944 13585 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc482211cb2b4a05bfcc386610cf864d" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40839 } }
I20260808 02:16:45.184036 13585 leader_election.cc:302] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc482211cb2b4a05bfcc386610cf864d; no voters:
I20260808 02:16:45.184198 13585 leader_election.cc:288] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:45.184233 13597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:45.184365 13597 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [term 1 LEADER]: Becoming Leader. State: Replica: fc482211cb2b4a05bfcc386610cf864d, State: Running, Role: LEADER
I20260808 02:16:45.184458 13585 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:45.184446 13597 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc482211cb2b4a05bfcc386610cf864d" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40839 } }
I20260808 02:16:45.184882 13597 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc482211cb2b4a05bfcc386610cf864d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc482211cb2b4a05bfcc386610cf864d" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40839 } } }
I20260808 02:16:45.184962 13597 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:45.185086 13597 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc482211cb2b4a05bfcc386610cf864d. Latest consensus state: current_term: 1 leader_uuid: "fc482211cb2b4a05bfcc386610cf864d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc482211cb2b4a05bfcc386610cf864d" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40839 } } }
I20260808 02:16:45.185150 13597 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d [sys.catalog]: This master's current role is: LEADER
W20260808 02:16:45.186607 13617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:16:45.186666 13617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:16:45.186777 13605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:45.186949 13605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:45.187327 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.177589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50878 (local address 127.19.58.254:40839)
0808 02:16:45.177762 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.177769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.177792 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:16:45.178227 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.178234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.178276 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.178392 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:45.178995 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.179959 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.180775 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.181198 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.181286 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.183619 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:45.183623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:45.183623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:45.183635 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:45.185690 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:45.186060 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:45.186063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:45.186063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:45.186162 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:45.186370 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:45.186371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:45.186371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:45.186669 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:45.186851 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.186988 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.187128 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260808 02:16:45.188663 13605 catalog_manager.cc:1347] Generated new cluster ID: 36872cc558394d9cadb8b0ea0cc4b3b9
I20260808 02:16:45.188722 13605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:45.196550 13605 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:45.197074 13605 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:45.198946 13605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d: Generated new TSK 0
I20260808 02:16:45.199108 13605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:16:45.232672 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40839
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:45.313776 13627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:45.314002 13627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:45.314034 13627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:45.315172 13627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:45.315212 13627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:45.315246 13627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:16:45.316738 13627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:45.316990 13627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:45.317201 13627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:45.321931 13635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:45.321992 13640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:45.321940 13634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:45.322079 13627 server_base.cc:1034] running on GCE node
I20260808 02:16:45.322322 13627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:45.322523 13627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:45.323658 13627 hybrid_clock.cc:648] HybridClock initialized: now 1786155405323647 us; error 33 us; skew 500 ppm
Aug 08 02:16:45 dist-test-slave-1l3l krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155405, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:45.326054 13627 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:16:45.326462 13627 webserver.cc:460] Webserver started at http://127.19.58.193:46761/ using document root <none> and password file <none>
I20260808 02:16:45.326664 13627 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:45.326707 13627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:45.326823 13627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:45.327849 13627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "2a566bd48d494255b6dfa6cb0cdd519c"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "ddbdbcbd29e6fee17979c0a9d78da43b"
server_key_iv: "e0ee45faba980d82116cac0feef2156b"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.328172 13627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "2a566bd48d494255b6dfa6cb0cdd519c"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "ddbdbcbd29e6fee17979c0a9d78da43b"
server_key_iv: "e0ee45faba980d82116cac0feef2156b"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.329346 13627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:16:45.330065 13649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.330252 13627 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:16:45.330376 13627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "2a566bd48d494255b6dfa6cb0cdd519c"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "ddbdbcbd29e6fee17979c0a9d78da43b"
server_key_iv: "e0ee45faba980d82116cac0feef2156b"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.330433 13627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:45.355994 13627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:45.357874 13627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:45.358006 13627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:45.358254 13627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:45.358584 13627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:45.358619 13627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.358649 13627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:45.358666 13627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.364295 13627 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:33271
I20260808 02:16:45.364362 13810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:33271 every 8 connection(s)
I20260808 02:16:45.364656 13627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:45.368125 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 13627
I20260808 02:16:45.368220 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:16:45.368395 19691 external_mini_cluster.cc:1428] Setting key f797969703ccd4cb5353ea83fda78e11
Aug 08 02:16:45 dist-test-slave-1l3l krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155405, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:45.373500 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.365295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:49011 (local address 127.19.58.254:40839)
0808 02:16:45.365337 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.365339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.365564 (+ 225us) server_negotiation.cc:407] Connection header received
0808 02:16:45.366059 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.366060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.366073 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.366127 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:45.367033 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.367553 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.368640 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.368906 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.368966 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.370996 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:45.370997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:45.370998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:45.371005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:45.372266 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:45.372638 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:45.372639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:45.372639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:45.372692 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:45.372910 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:45.372911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:45.372911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:45.373017 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:45.373073 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.373323 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.373410 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:16:45.373658 13813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.365268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40839 (local address 127.19.58.193:49011)
0808 02:16:45.365449 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.365485 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:45.365966 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:45.366196 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:45.366200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:45.366744 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:45.366940 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.366945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.367636 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.367637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.368554 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.368557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.368979 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.368980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.369109 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.369558 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:45.369565 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:45.370910 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:45.372339 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:45.372346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:45.372348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:45.372570 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:45.372747 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:45.372748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:45.372748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:45.372849 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:45.373071 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:45.373078 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:45.373209 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 02:16:45.373294 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:16:45.373451 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260808 02:16:45.374104 13811 heartbeater.cc:347] Connected to a master server at 127.19.58.254:40839
I20260808 02:16:45.374188 13811 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:45.374349 13811 heartbeater.cc:511] Master 127.19.58.254:40839 requested a full tablet report, sending...
I20260808 02:16:45.374910 13535 ts_manager.cc:194] Registered new tserver with Master: 2a566bd48d494255b6dfa6cb0cdd519c (127.19.58.193:33271)
I20260808 02:16:45.375845 13535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:49011
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:16:45.392287 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40839
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:45.471808 13820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:45.472030 13820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:45.472064 13820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:45.473210 13820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:45.473251 13820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:45.473286 13820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:16:45.474691 13820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:45.474923 13820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:45.475132 13820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:45.479921 13833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:45.479951 13830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:45.479933 13820 server_base.cc:1034] running on GCE node
W20260808 02:16:45.479960 13829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:45.480252 13820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:45.480453 13820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:45.481577 13820 hybrid_clock.cc:648] HybridClock initialized: now 1786155405481568 us; error 28 us; skew 500 ppm
Aug 08 02:16:45 dist-test-slave-1l3l krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155405, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:45.484007 13820 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:16:45.484411 13820 webserver.cc:460] Webserver started at http://127.19.58.194:38957/ using document root <none> and password file <none>
I20260808 02:16:45.484594 13820 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:45.484632 13820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:45.484724 13820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:45.485729 13820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "282dfc1c02e74adcb46ee668965b4399"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "f8547df0c009c958600dc058e472e0ea"
server_key_iv: "7498edbf625bc1edee44dcf23f29d3d4"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.486028 13820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "282dfc1c02e74adcb46ee668965b4399"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "f8547df0c009c958600dc058e472e0ea"
server_key_iv: "7498edbf625bc1edee44dcf23f29d3d4"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.487208 13820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:16:45.487855 13843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.488039 13820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:45.488122 13820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "282dfc1c02e74adcb46ee668965b4399"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "f8547df0c009c958600dc058e472e0ea"
server_key_iv: "7498edbf625bc1edee44dcf23f29d3d4"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.488175 13820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:45.498267 13820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:45.500499 13820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:45.500614 13820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:45.500801 13820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:45.501075 13820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:45.501104 13820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.501125 13820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:45.501137 13820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.506358 13820 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:41109
I20260808 02:16:45.506443 14000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:41109 every 8 connection(s)
I20260808 02:16:45.506696 13820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:16:45.507102 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 13820
I20260808 02:16:45.507211 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:16:45.507437 19691 external_mini_cluster.cc:1428] Setting key d27e57daea23e3724a27ea72ce58cac0
Aug 08 02:16:45 dist-test-slave-1l3l krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155405, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:45.515993 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.507443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:59381 (local address 127.19.58.254:40839)
0808 02:16:45.507495 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.507497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.507756 (+ 259us) server_negotiation.cc:407] Connection header received
0808 02:16:45.508312 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.508313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.508325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.508368 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:45.509302 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.509801 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.510780 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.510989 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.511046 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.513335 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:45.513337 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:45.513337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:45.513345 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:45.514586 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:45.515052 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:45.515053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:45.515053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:45.515098 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:45.515363 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:45.515363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:45.515363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:45.515455 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:45.515498 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.515856 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.515926 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:16:45.516242 14003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.507432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40839 (local address 127.19.58.194:59381)
0808 02:16:45.507609 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.507656 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:45.508214 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:45.508456 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:45.508460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:45.509017 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:45.509205 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.509210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.509885 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.509886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.510693 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.510695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.511059 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.511060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.511191 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.511738 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:45.511748 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:45.513241 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:45.514664 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:45.514672 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:45.514674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:45.514981 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:45.515154 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:45.515155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:45.515155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:45.515288 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:45.515508 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:45.515515 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:45.515679 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 02:16:45.515843 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 02:16:45.516020 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
I20260808 02:16:45.516776 14001 heartbeater.cc:347] Connected to a master server at 127.19.58.254:40839
I20260808 02:16:45.516865 14001 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:45.517015 14001 heartbeater.cc:511] Master 127.19.58.254:40839 requested a full tablet report, sending...
I20260808 02:16:45.517334 13535 ts_manager.cc:194] Registered new tserver with Master: 282dfc1c02e74adcb46ee668965b4399 (127.19.58.194:41109)
I20260808 02:16:45.517815 13535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:59381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:16:45.533591 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40839
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:45.613463 14011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:45.613648 14011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:45.613669 14011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:45.614754 14011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:45.614795 14011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:45.614820 14011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:16:45.616217 14011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:40839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:45.616423 14011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:45.616627 14011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:45.621212 14016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:45.621235 14017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:45.621234 14019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:45.621364 14011 server_base.cc:1034] running on GCE node
I20260808 02:16:45.621621 14011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:45.621829 14011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:45.622970 14011 hybrid_clock.cc:648] HybridClock initialized: now 1786155405622960 us; error 39 us; skew 500 ppm
Aug 08 02:16:45 dist-test-slave-1l3l krb5kdc[13467](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155405, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:45.625280 14011 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:16:45.625682 14011 webserver.cc:460] Webserver started at http://127.19.58.195:40347/ using document root <none> and password file <none>
I20260808 02:16:45.625869 14011 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:45.625919 14011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:45.626040 14011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:45.626994 14011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "b8dd21e01154459391d34d38a9a90030"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "ab0c4be2bd1b9557886e928baab194ce"
server_key_iv: "3c6852d32ea0247227dd30cf5cc21b39"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.627307 14011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "b8dd21e01154459391d34d38a9a90030"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "ab0c4be2bd1b9557886e928baab194ce"
server_key_iv: "3c6852d32ea0247227dd30cf5cc21b39"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.628724 14011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:16:45.629385 14028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.629590 14011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:16:45.629685 14011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "b8dd21e01154459391d34d38a9a90030"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "ab0c4be2bd1b9557886e928baab194ce"
server_key_iv: "3c6852d32ea0247227dd30cf5cc21b39"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.629758 14011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:45.638823 14011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:45.640677 14011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:45.640789 14011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:45.640969 14011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:45.641208 14011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:45.641237 14011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.641256 14011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:45.641273 14011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.646817 14011 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:46339
I20260808 02:16:45.646863 14190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:46339 every 8 connection(s)
I20260808 02:16:45.647152 14011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:16:45.648053 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 14011
I20260808 02:16:45.648161 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:16:45.648329 19691 external_mini_cluster.cc:1428] Setting key 812661c89731bf7da244b8a1809bbee4
Aug 08 02:16:45 dist-test-slave-1l3l krb5kdc[13467](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155405, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:45.655874 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.647795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:42517 (local address 127.19.58.254:40839)
0808 02:16:45.647847 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.647849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.648054 (+ 205us) server_negotiation.cc:407] Connection header received
0808 02:16:45.648592 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.648593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.648606 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.648659 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:45.649554 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.650066 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.651013 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.651216 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.651272 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.653363 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:45.653364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:45.653364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:45.653371 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:45.654625 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:45.654982 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:45.654984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:45.654984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:45.655036 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:45.655250 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:45.655251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:45.655251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:45.655369 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:45.655429 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.655681 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.655783 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:16:45.656041 14194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.647779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40839 (local address 127.19.58.195:42517)
0808 02:16:45.647950 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.647983 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:45.648496 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:45.648726 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:45.648730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:45.649276 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:45.649464 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.649469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.650146 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.650147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.650935 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.650937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.651284 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.651285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.651411 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.651919 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:45.651928 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:45.653274 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:45.654699 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:45.654705 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:45.654707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:45.654918 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:45.655091 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:45.655091 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:45.655092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:45.655194 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:45.655436 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:45.655443 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:45.655567 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 02:16:45.655651 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 02:16:45.655833 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260808 02:16:45.656448 14191 heartbeater.cc:347] Connected to a master server at 127.19.58.254:40839
I20260808 02:16:45.656533 14191 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:45.656687 14191 heartbeater.cc:511] Master 127.19.58.254:40839 requested a full tablet report, sending...
I20260808 02:16:45.656991 13535 ts_manager.cc:194] Registered new tserver with Master: b8dd21e01154459391d34d38a9a90030 (127.19.58.195:46339)
I20260808 02:16:45.657346 13535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:42517
I20260808 02:16:45.659796 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:16:45.666512 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.661139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50884 (local address 127.19.58.254:40839)
0808 02:16:45.661184 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.661185 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.661329 (+ 144us) server_negotiation.cc:407] Connection header received
0808 02:16:45.661352 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.661353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.661362 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.661414 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:45.661998 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.662480 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.663195 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.663363 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.663419 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.664516 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:45.664517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:45.664518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:45.664523 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:45.665667 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:45.665977 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:45.665978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:45.665978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:45.666021 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:45.666142 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:45.666142 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:45.666142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:45.666243 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:45.666312 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.666387 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.666454 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 02:16:45.667855 13535 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:16:45.669349 13535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:16:45.675494 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.671476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:46339 (local address 127.0.0.1:43742)
0808 02:16:45.671634 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.671642 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:45.671687 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:45.672116 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:45.672117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:45.672119 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:45.672157 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.672158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.673439 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.673440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.674294 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.674297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.675229 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.675230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.675289 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.675293 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:45.675352 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:16:45.675395 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":1,"thread_start_us":90,"threads_started":1}
I20260808 02:16:45.675530 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.671438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55040 (local address 127.19.58.194:41109)
0808 02:16:45.671677 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.671682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.671693 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:16:45.671807 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.671809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.671827 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.671916 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:45.672165 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.673304 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.674547 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.675207 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.675272 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.675336 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.675419 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.675460 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":96,"threads_started":1}
I20260808 02:16:45.675571 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.671415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47992 (local address 127.19.58.193:33271)
0808 02:16:45.671919 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.671924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.671934 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:16:45.671972 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.671974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.671994 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.672064 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:45.672349 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.673116 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.674588 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.675054 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.675116 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.675176 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.675446 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.675492 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":119,"threads_started":1}
I20260808 02:16:45.675506 14204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.671270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:41109 (local address 127.0.0.1:55040)
0808 02:16:45.671622 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.671633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:45.671687 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:45.672011 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:45.672014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:45.672016 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:45.672085 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.672087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.673413 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.673414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.674467 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.674470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.675272 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.675273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.675338 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.675341 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:45.675402 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 02:16:45.675439 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":74,"threads_started":1}
I20260808 02:16:45.675902 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.671698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43742 (local address 127.19.58.195:46339)
0808 02:16:45.671912 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.671916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.671925 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:16:45.671959 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.671961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.671975 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.672051 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:45.672308 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.673355 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.674379 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.675067 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.675136 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.675190 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.675774 (+ 584us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.675817 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":87,"threads_started":1}
I20260808 02:16:45.675657 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.671262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:33271 (local address 127.0.0.1:47992)
0808 02:16:45.671412 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.671451 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:45.671517 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:45.672226 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:45.672226 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:45.672229 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:45.672272 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.672273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.673209 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.673210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.674509 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.674511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.675131 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.675132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.675209 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.675221 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 02:16:45.675285 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 02:16:45.675351 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":61,"threads_started":1}
I20260808 02:16:45.676546 13721 tablet_service.cc:1467] Processing CreateTablet for tablet 7061facae8334984a1c297391fc6e32e (DEFAULT_TABLE table=test-table [id=4a349d8d94da4b84927ccf3694ec87d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:45.676558 14102 tablet_service.cc:1467] Processing CreateTablet for tablet 7061facae8334984a1c297391fc6e32e (DEFAULT_TABLE table=test-table [id=4a349d8d94da4b84927ccf3694ec87d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:45.676546 13908 tablet_service.cc:1467] Processing CreateTablet for tablet 7061facae8334984a1c297391fc6e32e (DEFAULT_TABLE table=test-table [id=4a349d8d94da4b84927ccf3694ec87d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:45.676906 13908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7061facae8334984a1c297391fc6e32e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:45.676909 13721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7061facae8334984a1c297391fc6e32e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:45.676923 14102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7061facae8334984a1c297391fc6e32e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:45.679311 14213 tablet_bootstrap.cc:492] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: Bootstrap starting.
I20260808 02:16:45.680088 14216 tablet_bootstrap.cc:492] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c: Bootstrap starting.
I20260808 02:16:45.680310 14213 tablet_bootstrap.cc:654] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:45.680752 14216 tablet_bootstrap.cc:654] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:45.680877 14213 log.cc:824] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:45.681313 14216 log.cc:824] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:45.681653 14213 tablet_bootstrap.cc:492] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: No bootstrap required, opened a new log
I20260808 02:16:45.681644 14215 tablet_bootstrap.cc:492] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: Bootstrap starting.
I20260808 02:16:45.681718 14213 ts_tablet_manager.cc:1397] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:45.682001 14216 tablet_bootstrap.cc:492] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c: No bootstrap required, opened a new log
I20260808 02:16:45.682060 14216 ts_tablet_manager.cc:1397] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:45.682250 14215 tablet_bootstrap.cc:654] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:45.682788 14215 log.cc:824] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:45.683529 14215 tablet_bootstrap.cc:492] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: No bootstrap required, opened a new log
I20260808 02:16:45.683600 14215 ts_tablet_manager.cc:1397] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:45.683634 14216 raft_consensus.cc:348] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 } } peers { permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 } } peers { permanent_uuid: "282dfc1c02e74adcb46ee668965b4399" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41109 } }
I20260808 02:16:45.683820 14216 raft_consensus.cc:374] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:45.683864 14216 raft_consensus.cc:729] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a566bd48d494255b6dfa6cb0cdd519c, State: Initialized, Role: FOLLOWER
I20260808 02:16:45.683815 14213 raft_consensus.cc:348] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 } } peers { permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 } } peers { permanent_uuid: "282dfc1c02e74adcb46ee668965b4399" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41109 } }
I20260808 02:16:45.683979 14213 raft_consensus.cc:374] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:45.684017 14213 raft_consensus.cc:729] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8dd21e01154459391d34d38a9a90030, State: Initialized, Role: FOLLOWER
I20260808 02:16:45.683982 14216 consensus_queue.cc:260] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 } } peers { permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 } } peers { permanent_uuid: "282dfc1c02e74adcb46ee668965b4399" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41109 } }
I20260808 02:16:45.684144 14213 consensus_queue.cc:260] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 } } peers { permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 } } peers { permanent_uuid: "282dfc1c02e74adcb46ee668965b4399" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41109 } }
I20260808 02:16:45.684255 14216 ts_tablet_manager.cc:1428] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:45.684312 13811 heartbeater.cc:503] Master 127.19.58.254:40839 was elected leader, sending a full tablet report...
I20260808 02:16:45.684418 14213 ts_tablet_manager.cc:1428] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:45.684532 14191 heartbeater.cc:503] Master 127.19.58.254:40839 was elected leader, sending a full tablet report...
I20260808 02:16:45.685575 14215 raft_consensus.cc:348] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 } } peers { permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 } } peers { permanent_uuid: "282dfc1c02e74adcb46ee668965b4399" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41109 } }
I20260808 02:16:45.685725 14215 raft_consensus.cc:374] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:45.685750 14215 raft_consensus.cc:729] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282dfc1c02e74adcb46ee668965b4399, State: Initialized, Role: FOLLOWER
I20260808 02:16:45.685863 14215 consensus_queue.cc:260] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 } } peers { permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 } } peers { permanent_uuid: "282dfc1c02e74adcb46ee668965b4399" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41109 } }
I20260808 02:16:45.686092 14215 ts_tablet_manager.cc:1428] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:16:45.686141 14001 heartbeater.cc:503] Master 127.19.58.254:40839 was elected leader, sending a full tablet report...
W20260808 02:16:45.757254 14002 tablet.cc:2369] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:45.782362 14225 raft_consensus.cc:482] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:45.782516 14225 raft_consensus.cc:504] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 } } peers { permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 } } peers { permanent_uuid: "282dfc1c02e74adcb46ee668965b4399" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41109 } }
I20260808 02:16:45.782907 14225 leader_election.cc:288] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a566bd48d494255b6dfa6cb0cdd519c (127.19.58.193:33271), b8dd21e01154459391d34d38a9a90030 (127.19.58.195:46339)
I20260808 02:16:45.786466 14003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.783086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:33271 (local address 127.19.58.194:46795)
0808 02:16:45.783138 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.783150 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:45.783196 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:45.783405 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:45.783406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:45.783409 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:45.783486 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.783487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.784294 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.784294 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.785410 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.785412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.786179 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.786181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.786276 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.786281 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:16:45.786334 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 02:16:45.786387 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 02:16:45.786464 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.783168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:46795 (local address 127.19.58.193:33271)
0808 02:16:45.783213 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.783215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.783253 (+ 38us) server_negotiation.cc:407] Connection header received
0808 02:16:45.783283 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.783284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.783293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.783342 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:45.783551 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.784224 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.785533 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.786093 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.786155 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.786164 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.786351 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.786390 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:16:45.786885 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.783440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:60989 (local address 127.19.58.195:46339)
0808 02:16:45.783579 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.783582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.783589 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:16:45.783613 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.783615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.783624 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.783674 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:45.783887 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.784588 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.785832 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.786350 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.786392 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.786409 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.786764 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.786812 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260808 02:16:45.786862 13750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7061facae8334984a1c297391fc6e32e" candidate_uuid: "282dfc1c02e74adcb46ee668965b4399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" is_pre_election: true
I20260808 02:16:45.787022 13750 raft_consensus.cc:2454] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 282dfc1c02e74adcb46ee668965b4399 in term 0.
I20260808 02:16:45.787078 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.783203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:46339 (local address 127.19.58.194:60989)
0808 02:16:45.783376 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.783385 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:45.783454 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:45.783693 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:45.783694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:45.783695 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:45.783788 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.783790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.784657 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.784658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.785743 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:45.785746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.786468 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.786469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:45.786677 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.786682 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:16:45.786965 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 02:16:45.787007 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":111,"threads_started":1}
I20260808 02:16:45.787295 13848 leader_election.cc:302] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282dfc1c02e74adcb46ee668965b4399, 2a566bd48d494255b6dfa6cb0cdd519c; no voters:
I20260808 02:16:45.787519 14128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7061facae8334984a1c297391fc6e32e" candidate_uuid: "282dfc1c02e74adcb46ee668965b4399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8dd21e01154459391d34d38a9a90030" is_pre_election: true
I20260808 02:16:45.787642 14128 raft_consensus.cc:2454] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 282dfc1c02e74adcb46ee668965b4399 in term 0.
I20260808 02:16:45.787775 14225 raft_consensus.cc:2781] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:16:45.787845 14225 raft_consensus.cc:482] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:16:45.787878 14225 raft_consensus.cc:3037] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:45.788606 14225 raft_consensus.cc:504] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 } } peers { permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 } } peers { permanent_uuid: "282dfc1c02e74adcb46ee668965b4399" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41109 } }
I20260808 02:16:45.788794 14225 leader_election.cc:288] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [CANDIDATE]: Term 1 election: Requested vote from peers 2a566bd48d494255b6dfa6cb0cdd519c (127.19.58.193:33271), b8dd21e01154459391d34d38a9a90030 (127.19.58.195:46339)
I20260808 02:16:45.788852 13750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7061facae8334984a1c297391fc6e32e" candidate_uuid: "282dfc1c02e74adcb46ee668965b4399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a566bd48d494255b6dfa6cb0cdd519c"
I20260808 02:16:45.788938 13750 raft_consensus.cc:3037] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:45.788947 14128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7061facae8334984a1c297391fc6e32e" candidate_uuid: "282dfc1c02e74adcb46ee668965b4399" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8dd21e01154459391d34d38a9a90030"
I20260808 02:16:45.789016 14128 raft_consensus.cc:3037] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:45.789705 14128 raft_consensus.cc:2454] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282dfc1c02e74adcb46ee668965b4399 in term 1.
I20260808 02:16:45.789855 13848 leader_election.cc:302] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 282dfc1c02e74adcb46ee668965b4399, b8dd21e01154459391d34d38a9a90030; no voters:
I20260808 02:16:45.789927 13750 raft_consensus.cc:2454] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 282dfc1c02e74adcb46ee668965b4399 in term 1.
I20260808 02:16:45.789950 14225 raft_consensus.cc:2781] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:45.790096 14225 raft_consensus.cc:686] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 1 LEADER]: Becoming Leader. State: Replica: 282dfc1c02e74adcb46ee668965b4399, State: Running, Role: LEADER
I20260808 02:16:45.790201 14225 consensus_queue.cc:237] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 } } peers { permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 } } peers { permanent_uuid: "282dfc1c02e74adcb46ee668965b4399" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41109 } }
I20260808 02:16:45.791023 13534 catalog_manager.cc:5495] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 reported cstate change: term changed from 0 to 1, leader changed from <none> to 282dfc1c02e74adcb46ee668965b4399 (127.19.58.194). New cstate: current_term: 1 leader_uuid: "282dfc1c02e74adcb46ee668965b4399" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "282dfc1c02e74adcb46ee668965b4399" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 41109 } health_report { overall_health: HEALTHY } } }
I20260808 02:16:45.819307 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.816748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55050 (local address 127.19.58.194:41109)
0808 02:16:45.816795 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:45.816797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:45.816806 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:16:45.816884 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:45.816885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:45.816902 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:45.816950 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:45.817078 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.817538 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.818600 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:45.818790 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:45.818842 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:45.819011 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:45.819094 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:45.819160 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:45.819220 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 02:16:45.819229 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:16:45.821262 14128 raft_consensus.cc:1261] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030 [term 1 FOLLOWER]: Refusing update from remote peer 282dfc1c02e74adcb46ee668965b4399: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:45.821310 13750 raft_consensus.cc:1261] T 7061facae8334984a1c297391fc6e32e P 2a566bd48d494255b6dfa6cb0cdd519c [term 1 FOLLOWER]: Refusing update from remote peer 282dfc1c02e74adcb46ee668965b4399: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:45.821622 14225 consensus_queue.cc:1035] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8dd21e01154459391d34d38a9a90030" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:45.821753 14230 consensus_queue.cc:1035] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a566bd48d494255b6dfa6cb0cdd519c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 33271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:45.825037 14233 mvcc.cc:205] Tried to move back new op lower bound from 7316092542241374208 to 7316092542117212160. Current Snapshot: MvccSnapshot[applied={T|T < 7316092542241374208 or (T in {7316092542241374208})}]
I20260808 02:16:45.826363 14235 mvcc.cc:205] Tried to move back new op lower bound from 7316092542241374208 to 7316092542117212160. Current Snapshot: MvccSnapshot[applied={T|T < 7316092542241374208 or (T in {7316092542241374208})}]
I20260808 02:16:45.826615 13534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:45.826700 13534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:45.827410 13534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc482211cb2b4a05bfcc386610cf864d: Sending DeleteTablet for 3 replicas of tablet 7061facae8334984a1c297391fc6e32e
I20260808 02:16:45.827771 14102 tablet_service.cc:1514] Processing DeleteTablet for tablet 7061facae8334984a1c297391fc6e32e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:45 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:43742
I20260808 02:16:45.827809 13721 tablet_service.cc:1514] Processing DeleteTablet for tablet 7061facae8334984a1c297391fc6e32e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:45 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:47992
I20260808 02:16:45.828027 13908 tablet_service.cc:1514] Processing DeleteTablet for tablet 7061facae8334984a1c297391fc6e32e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:45 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:55040
I20260808 02:16:45.828241 14247 tablet_replica.cc:330] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: stopping tablet replica
I20260808 02:16:45.828338 14247 raft_consensus.cc:2229] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:16:45.828439 14247 raft_consensus.cc:2258] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:45.828665 14245 tablet_replica.cc:330] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: stopping tablet replica
I20260808 02:16:45.828752 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 13627
I20260808 02:16:45.831091 14247 ts_tablet_manager.cc:1905] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:45.831418 14245 raft_consensus.cc:2229] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:45.831625 14245 raft_consensus.cc:2258] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:45.833299 14247 ts_tablet_manager.cc:1918] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:45.833390 14247 log.cc:1196] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/7061facae8334984a1c297391fc6e32e
I20260808 02:16:45.833829 14247 ts_tablet_manager.cc:1939] T 7061facae8334984a1c297391fc6e32e P 282dfc1c02e74adcb46ee668965b4399: Deleting consensus metadata
I20260808 02:16:45.835757 14245 ts_tablet_manager.cc:1905] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:45.837313 14245 ts_tablet_manager.cc:1918] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:45.837404 14245 log.cc:1196] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/7061facae8334984a1c297391fc6e32e
I20260808 02:16:45.837678 14245 ts_tablet_manager.cc:1939] T 7061facae8334984a1c297391fc6e32e P b8dd21e01154459391d34d38a9a90030: Deleting consensus metadata
I20260808 02:16:45.843927 13515 catalog_manager.cc:4841] TS 282dfc1c02e74adcb46ee668965b4399 (127.19.58.194:41109): tablet 7061facae8334984a1c297391fc6e32e (table test-table [id=4a349d8d94da4b84927ccf3694ec87d1]) successfully deleted
I20260808 02:16:45.844293 13516 catalog_manager.cc:4841] TS b8dd21e01154459391d34d38a9a90030 (127.19.58.195:46339): tablet 7061facae8334984a1c297391fc6e32e (table test-table [id=4a349d8d94da4b84927ccf3694ec87d1]) successfully deleted
W20260808 02:16:45.850977 13516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:33271 (error 108)
I20260808 02:16:45.851053 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 13820
I20260808 02:16:45.851455 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:45.851347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:33271 (local address 127.0.0.1:47996)
0808 02:16:45.851392 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:45.851422 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:33271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 02:16:45.851555 13516 catalog_manager.cc:4584] TS 2a566bd48d494255b6dfa6cb0cdd519c (127.19.58.193:33271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7061facae8334984a1c297391fc6e32e: Network error: Client connection negotiation failed: client connection to 127.19.58.193:33271: connect: Connection refused (error 111)
I20260808 02:16:45.861411 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 14011
I20260808 02:16:45.869652 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 13489
2026-08-08T02:16:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2053 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T02:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:45Z Disabled control of system clock
I20260808 02:16:45.902381 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42159
--webserver_interface=127.19.58.254
--webserver_port=0
--builtin_ntp_servers=127.19.58.212:42203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:42159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:45.981657 14253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:45.981855 14253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:45.981881 14253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:45.982985 14253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:45.983024 14253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:45.983035 14253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:45.983058 14253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:45.983081 14253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:45.984417 14253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:42159
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:45.984637 14253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:45.984906 14253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:45.989437 14265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:45.989516 14253 server_base.cc:1034] running on GCE node
W20260808 02:16:45.989501 14263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:45.989439 14262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:45.989831 14253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:45.990108 14253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:45.991255 14253 hybrid_clock.cc:648] HybridClock initialized: now 1786155405991234 us; error 36 us; skew 500 ppm
I20260808 02:16:45.992614 14253 webserver.cc:460] Webserver started at http://127.19.58.254:42641/ using document root <none> and password file <none>
I20260808 02:16:45.992843 14253 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:45.992893 14253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:45.993018 14253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:45.994055 14253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "b306dbdd8caa49b8b40e7590e8ded6a7"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "f1d67514fb433787518a72a00ed8263f"
server_key_iv: "fa6ac15e1c189215412419ee5c99c06b"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.994400 14253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "b306dbdd8caa49b8b40e7590e8ded6a7"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "f1d67514fb433787518a72a00ed8263f"
server_key_iv: "fa6ac15e1c189215412419ee5c99c06b"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.995659 14253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:16:45.996417 14274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:45.996562 14253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:45.996651 14253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "b306dbdd8caa49b8b40e7590e8ded6a7"
format_stamp: "Formatted at 2026-08-08 02:16:45 on dist-test-slave-1l3l"
server_key: "f1d67514fb433787518a72a00ed8263f"
server_key_iv: "fa6ac15e1c189215412419ee5c99c06b"
server_key_version: "encryptionkey@0"
I20260808 02:16:45.996726 14253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:46.023265 14253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:46.023613 14253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:46.023785 14253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:46.027590 14253 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:42159
I20260808 02:16:46.027660 14347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:42159 every 8 connection(s)
I20260808 02:16:46.028007 14253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:46.028997 14349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:46.031388 14349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7: Bootstrap starting.
I20260808 02:16:46.032004 14349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:46.032330 14349 log.cc:824] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:46.033052 14349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7: No bootstrap required, opened a new log
I20260808 02:16:46.034658 14349 raft_consensus.cc:348] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b306dbdd8caa49b8b40e7590e8ded6a7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42159 } }
I20260808 02:16:46.034798 14349 raft_consensus.cc:374] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:46.034821 14349 raft_consensus.cc:729] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b306dbdd8caa49b8b40e7590e8ded6a7, State: Initialized, Role: FOLLOWER
I20260808 02:16:46.034968 14349 consensus_queue.cc:260] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b306dbdd8caa49b8b40e7590e8ded6a7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42159 } }
I20260808 02:16:46.035032 14349 raft_consensus.cc:388] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:46.035053 14349 raft_consensus.cc:482] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:46.035097 14349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:46.035632 14349 raft_consensus.cc:504] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b306dbdd8caa49b8b40e7590e8ded6a7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42159 } }
I20260808 02:16:46.035749 14349 leader_election.cc:302] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b306dbdd8caa49b8b40e7590e8ded6a7; no voters:
I20260808 02:16:46.035921 14349 leader_election.cc:288] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:46.035992 14352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:46.036160 14352 raft_consensus.cc:686] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [term 1 LEADER]: Becoming Leader. State: Replica: b306dbdd8caa49b8b40e7590e8ded6a7, State: Running, Role: LEADER
I20260808 02:16:46.036172 14349 sys_catalog.cc:564] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:46.036275 14352 consensus_queue.cc:237] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b306dbdd8caa49b8b40e7590e8ded6a7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42159 } }
I20260808 02:16:46.036628 14354 sys_catalog.cc:455] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b306dbdd8caa49b8b40e7590e8ded6a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b306dbdd8caa49b8b40e7590e8ded6a7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42159 } } }
I20260808 02:16:46.036702 14354 sys_catalog.cc:458] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:46.036733 14355 sys_catalog.cc:455] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b306dbdd8caa49b8b40e7590e8ded6a7. Latest consensus state: current_term: 1 leader_uuid: "b306dbdd8caa49b8b40e7590e8ded6a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b306dbdd8caa49b8b40e7590e8ded6a7" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42159 } } }
I20260808 02:16:46.036813 14355 sys_catalog.cc:458] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:46.036864 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 14253
I20260808 02:16:46.036981 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:46.037166 19691 external_mini_cluster.cc:1428] Setting key dbfc5f3ed1691dad7ba0588a24f20c15
I20260808 02:16:46.037729 14372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:46.038399 14372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:46.040777 14372 catalog_manager.cc:1347] Generated new cluster ID: a23a5b813f7040fcae027e57c1490ad7
I20260808 02:16:46.040838 14372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:46.042559 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.037682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53864 (local address 127.19.58.254:42159)
0808 02:16:46.037931 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.037941 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.037974 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:16:46.038070 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.038075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.038128 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.038257 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:46.038472 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.039633 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.040733 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.041201 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.041309 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.041966 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:46.041970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:46.041970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:46.042140 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:46.042172 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:46.042182 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:46.042241 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.042308 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.042393 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":63,"threads_started":1}
I20260808 02:16:46.098269 14372 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:46.099061 14372 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:46.105769 14372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7: Generated new TSK 0
I20260808 02:16:46.105948 14372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:46.111934 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42159
--builtin_ntp_servers=127.19.58.212:42203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:46.192466 14390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:46.192694 14390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:46.192726 14390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:46.194016 14390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:46.194067 14390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:46.194103 14390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:16:46.195525 14390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:46.195784 14390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:46.196072 14390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:46.200918 14397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:46.200910 14399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:46.201046 14390 server_base.cc:1034] running on GCE node
W20260808 02:16:46.201191 14403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:46.201409 14390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:46.201640 14390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:46.202790 14390 hybrid_clock.cc:648] HybridClock initialized: now 1786155406202777 us; error 28 us; skew 500 ppm
I20260808 02:16:46.204056 14390 webserver.cc:460] Webserver started at http://127.19.58.193:37429/ using document root <none> and password file <none>
I20260808 02:16:46.204264 14390 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:46.204361 14390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:46.204525 14390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:46.205653 14390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "3159ef6791a74de0be98a7f6e9ad7a2c"
format_stamp: "Formatted at 2026-08-08 02:16:46 on dist-test-slave-1l3l"
server_key: "f233edb007956afd47da90716bcbbea1"
server_key_iv: "e6204a7ea7c772613e67974402f9d613"
server_key_version: "encryptionkey@0"
I20260808 02:16:46.206010 14390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "3159ef6791a74de0be98a7f6e9ad7a2c"
format_stamp: "Formatted at 2026-08-08 02:16:46 on dist-test-slave-1l3l"
server_key: "f233edb007956afd47da90716bcbbea1"
server_key_iv: "e6204a7ea7c772613e67974402f9d613"
server_key_version: "encryptionkey@0"
I20260808 02:16:46.207291 14390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:16:46.207963 14412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:46.208107 14390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:46.208189 14390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "3159ef6791a74de0be98a7f6e9ad7a2c"
format_stamp: "Formatted at 2026-08-08 02:16:46 on dist-test-slave-1l3l"
server_key: "f233edb007956afd47da90716bcbbea1"
server_key_iv: "e6204a7ea7c772613e67974402f9d613"
server_key_version: "encryptionkey@0"
I20260808 02:16:46.208252 14390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:46.224442 14390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:46.224681 14390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:46.224797 14390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:46.224990 14390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:46.225273 14390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:46.225313 14390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:46.225349 14390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:46.225373 14390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:46.230841 14390 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:37559
I20260808 02:16:46.230904 14577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:37559 every 8 connection(s)
I20260808 02:16:46.231209 14390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:46.236217 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.231974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:54881 (local address 127.19.58.254:42159)
0808 02:16:46.232039 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.232042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.232264 (+ 222us) server_negotiation.cc:407] Connection header received
0808 02:16:46.232728 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.232729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.232740 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.232788 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:46.233464 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.233966 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.234951 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.235128 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.235171 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.235976 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:46.235978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:46.235978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:46.235984 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:46.236001 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:46.236005 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:46.236051 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.236117 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.236148 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 02:16:46.236421 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.231958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42159 (local address 127.19.58.193:54881)
0808 02:16:46.232147 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.232183 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:46.232653 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:46.232896 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:46.232899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:46.232909 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:46.233401 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.233406 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.234073 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.234074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.234859 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.234861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.235184 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.235185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.235292 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.235803 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:46.235804 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:46.235838 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:46.235859 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:46.236054 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:46.236060 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:46.236063 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:46.236114 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 02:16:46.236213 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260808 02:16:46.236896 14580 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42159
I20260808 02:16:46.236979 14580 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:46.237161 14580 heartbeater.cc:511] Master 127.19.58.254:42159 requested a full tablet report, sending...
I20260808 02:16:46.237608 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 14390
I20260808 02:16:46.237637 14298 ts_manager.cc:194] Registered new tserver with Master: 3159ef6791a74de0be98a7f6e9ad7a2c (127.19.58.193:37559)
I20260808 02:16:46.237704 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:16:46.237915 19691 external_mini_cluster.cc:1428] Setting key d819c79a2dbf40d76df0ba5b41e1948b
I20260808 02:16:46.238405 14298 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.193:54881
I20260808 02:16:46.238986 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42159
--builtin_ntp_servers=127.19.58.212:42203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:46.318063 14585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:46.318262 14585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:46.318282 14585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:46.319383 14585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:46.319420 14585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:46.319445 14585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:16:46.320775 14585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:46.321002 14585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:46.321269 14585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:46.326000 14591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:46.326018 14592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:46.326018 14594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:46.326162 14585 server_base.cc:1034] running on GCE node
I20260808 02:16:46.326454 14585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:46.326670 14585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:46.327760 14585 hybrid_clock.cc:648] HybridClock initialized: now 1786155406327740 us; error 33 us; skew 500 ppm
I20260808 02:16:46.328945 14585 webserver.cc:460] Webserver started at http://127.19.58.194:46475/ using document root <none> and password file <none>
I20260808 02:16:46.329176 14585 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:46.329226 14585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:46.329355 14585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:46.330369 14585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "fcce1431ce2c4036b9f3cdc64f80d475"
format_stamp: "Formatted at 2026-08-08 02:16:46 on dist-test-slave-1l3l"
server_key: "0f6e38e6f67250c54960a712d221d2bb"
server_key_iv: "017154b162773168f8c8bbcdfeb187ec"
server_key_version: "encryptionkey@0"
I20260808 02:16:46.330699 14585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "fcce1431ce2c4036b9f3cdc64f80d475"
format_stamp: "Formatted at 2026-08-08 02:16:46 on dist-test-slave-1l3l"
server_key: "0f6e38e6f67250c54960a712d221d2bb"
server_key_iv: "017154b162773168f8c8bbcdfeb187ec"
server_key_version: "encryptionkey@0"
I20260808 02:16:46.331919 14585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:16:46.332587 14604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:46.332728 14585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:46.332808 14585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "fcce1431ce2c4036b9f3cdc64f80d475"
format_stamp: "Formatted at 2026-08-08 02:16:46 on dist-test-slave-1l3l"
server_key: "0f6e38e6f67250c54960a712d221d2bb"
server_key_iv: "017154b162773168f8c8bbcdfeb187ec"
server_key_version: "encryptionkey@0"
I20260808 02:16:46.332872 14585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:46.354676 14585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:46.354995 14585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:46.355144 14585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:46.355422 14585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:46.355777 14585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:46.355813 14585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:46.355842 14585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:46.355859 14585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:46.361160 14585 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:35133
I20260808 02:16:46.361234 14771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:35133 every 8 connection(s)
I20260808 02:16:46.361533 14585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:16:46.364109 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 14585
I20260808 02:16:46.364218 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:16:46.364413 19691 external_mini_cluster.cc:1428] Setting key 254412ccdc587aef634a8d38f80bf891
I20260808 02:16:46.365399 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42159
--builtin_ntp_servers=127.19.58.212:42203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 02:16:46.366863 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.362187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:54649 (local address 127.19.58.254:42159)
0808 02:16:46.362234 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.362236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.362483 (+ 247us) server_negotiation.cc:407] Connection header received
0808 02:16:46.362934 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.362935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.362944 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.363000 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:46.363614 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.364278 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.365565 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.365767 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.365798 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.366608 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:46.366609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:46.366610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:46.366614 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:46.366629 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:46.366633 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:46.366679 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.366757 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.366784 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:16:46.367096 14774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.362167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42159 (local address 127.19.58.194:54649)
0808 02:16:46.362356 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.362409 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:46.362857 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:46.363054 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:46.363057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:46.363065 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:46.363552 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.363557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.364381 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.364382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.365454 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.365458 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.365867 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.365867 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.365983 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.366481 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:46.366481 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:46.366509 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:46.366530 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:46.366685 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:46.366690 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:46.366693 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:46.366755 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 02:16:46.366855 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260808 02:16:46.367494 14772 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42159
I20260808 02:16:46.367578 14772 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:46.367777 14772 heartbeater.cc:511] Master 127.19.58.254:42159 requested a full tablet report, sending...
I20260808 02:16:46.368112 14298 ts_manager.cc:194] Registered new tserver with Master: fcce1431ce2c4036b9f3cdc64f80d475 (127.19.58.194:35133)
I20260808 02:16:46.368440 14298 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.194:54649
W20260808 02:16:46.445640 14776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:46.445844 14776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:46.445874 14776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:46.446964 14776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:46.447003 14776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:46.447036 14776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:16:46.448338 14776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:42159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:46.448563 14776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:46.448832 14776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:46.453459 14786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:46.453483 14787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:46.453483 14790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:46.453625 14776 server_base.cc:1034] running on GCE node
I20260808 02:16:46.453908 14776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:46.454139 14776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:46.455307 14776 hybrid_clock.cc:648] HybridClock initialized: now 1786155406455273 us; error 33 us; skew 500 ppm
I20260808 02:16:46.456472 14776 webserver.cc:460] Webserver started at http://127.19.58.195:38321/ using document root <none> and password file <none>
I20260808 02:16:46.456674 14776 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:46.456717 14776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:46.456843 14776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:46.457844 14776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "25fb7a89ec964aea910137969cea3e26"
format_stamp: "Formatted at 2026-08-08 02:16:46 on dist-test-slave-1l3l"
server_key: "8f4c041f6bb746597de439d9ca495122"
server_key_iv: "15a895cd003dfa702231303e79a91b85"
server_key_version: "encryptionkey@0"
I20260808 02:16:46.458204 14776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "25fb7a89ec964aea910137969cea3e26"
format_stamp: "Formatted at 2026-08-08 02:16:46 on dist-test-slave-1l3l"
server_key: "8f4c041f6bb746597de439d9ca495122"
server_key_iv: "15a895cd003dfa702231303e79a91b85"
server_key_version: "encryptionkey@0"
I20260808 02:16:46.459398 14776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:16:46.460079 14798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:46.460219 14776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:46.460304 14776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "25fb7a89ec964aea910137969cea3e26"
format_stamp: "Formatted at 2026-08-08 02:16:46 on dist-test-slave-1l3l"
server_key: "8f4c041f6bb746597de439d9ca495122"
server_key_iv: "15a895cd003dfa702231303e79a91b85"
server_key_version: "encryptionkey@0"
I20260808 02:16:46.460371 14776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:46.476593 14776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:46.476900 14776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:46.477032 14776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:46.477258 14776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:46.477592 14776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:46.477626 14776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:46.477648 14776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:46.477667 14776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:46.483168 14776 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:39889
I20260808 02:16:46.483212 14969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:39889 every 8 connection(s)
I20260808 02:16:46.483529 14776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:16:46.488569 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.484238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:38669 (local address 127.19.58.254:42159)
0808 02:16:46.484302 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.484304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.484557 (+ 253us) server_negotiation.cc:407] Connection header received
0808 02:16:46.485051 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.485053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.485065 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.485125 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:46.485783 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.486351 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.487256 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.487482 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.487533 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.488289 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:46.488290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:46.488291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:46.488295 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:46.488359 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:46.488363 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:46.488417 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.488480 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.488508 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 02:16:46.488725 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.484220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:42159 (local address 127.19.58.195:38669)
0808 02:16:46.484439 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.484473 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:46.484931 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:46.485184 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:46.485188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:46.485195 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:46.485695 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.485700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.486432 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.486433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.487184 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.487186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.487539 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.487540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.487640 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.488143 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:46.488143 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:46.488171 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:46.488192 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:46.488422 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:46.488424 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:46.488427 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:46.488474 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 02:16:46.488548 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260808 02:16:46.489161 14970 heartbeater.cc:347] Connected to a master server at 127.19.58.254:42159
I20260808 02:16:46.489267 14970 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:46.489442 14970 heartbeater.cc:511] Master 127.19.58.254:42159 requested a full tablet report, sending...
I20260808 02:16:46.489749 14298 ts_manager.cc:194] Registered new tserver with Master: 25fb7a89ec964aea910137969cea3e26 (127.19.58.195:39889)
I20260808 02:16:46.490091 14298 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.195:38669
I20260808 02:16:46.490787 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 14776
I20260808 02:16:46.490893 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:16:46.491062 19691 external_mini_cluster.cc:1428] Setting key a5662e35419d6c7357ce13f3e0637b08
I20260808 02:16:46.491941 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:16:46.496115 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.493278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.19.58.254:42159)
0808 02:16:46.493337 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.493339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.493415 (+ 76us) server_negotiation.cc:407] Connection header received
0808 02:16:46.493468 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.493469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.493480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.493526 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:46.493665 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.494171 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.494909 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.495129 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.495186 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.495840 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:46.495841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:46.495841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:46.495846 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:46.495860 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:46.495864 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:46.495951 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.496023 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.496032 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:16:46.497491 14298 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:16:46.498961 14298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:16:46.504637 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.500976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33962 (local address 127.19.58.195:39889)
0808 02:16:46.501394 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.501397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.501406 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:16:46.501441 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.501443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.501458 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.501543 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:46.501682 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.502511 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.503566 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.504261 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.504321 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.504409 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.504518 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.504554 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":95,"threads_started":1}
I20260808 02:16:46.504626 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.500890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:39889 (local address 127.0.0.1:33962)
0808 02:16:46.501161 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.501200 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:46.501262 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:46.501546 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:46.501549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:46.501551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:46.501615 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.501616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.502589 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.502590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.503483 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.503486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.504339 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.504340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.504420 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.504434 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 02:16:46.504500 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 02:16:46.504553 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":48,"thread_start_us":68,"threads_started":1}
I20260808 02:16:46.505139 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.501111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:37559 (local address 127.0.0.1:49878)
0808 02:16:46.501371 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.501380 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:46.501426 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:46.501636 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:46.501637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:46.501640 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:46.501683 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.501685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.502652 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.502653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.503884 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.503886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.504912 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.504914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.504982 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.504985 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:46.505031 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 02:16:46.505068 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":22,"thread_start_us":69,"threads_started":1}
I20260808 02:16:46.505164 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.501207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49878 (local address 127.19.58.193:37559)
0808 02:16:46.501357 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.501360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.501423 (+ 63us) server_negotiation.cc:407] Connection header received
0808 02:16:46.501490 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.501491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.501505 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.501571 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:46.501752 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.502569 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.503967 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.504817 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.504889 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.504979 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.505044 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.505085 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260808 02:16:46.505584 14877 tablet_service.cc:1467] Processing CreateTablet for tablet 5df2e76ee97842b2b49e6b67187e9559 (DEFAULT_TABLE table=test-table [id=80be4b8a70ac4a35afbcaf2ccab31576]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:46.505925 14982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.500912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35133 (local address 127.0.0.1:38926)
0808 02:16:46.501223 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.501231 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:46.501279 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:46.501546 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:46.501549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:46.501554 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:46.501618 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.501619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.503797 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.503798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.504834 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.504837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.505703 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.505704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.505767 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.505770 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:46.505824 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 02:16:46.505859 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":5,"thread_start_us":70,"threads_started":1}
I20260808 02:16:46.505988 14877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5df2e76ee97842b2b49e6b67187e9559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:46.505959 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.501020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38926 (local address 127.19.58.194:35133)
0808 02:16:46.501207 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.501211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.501276 (+ 65us) server_negotiation.cc:407] Connection header received
0808 02:16:46.501392 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.501395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.501412 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.501480 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:46.502592 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.503687 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.504930 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.505625 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.505694 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.505774 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.505838 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.505874 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":81,"threads_started":1}
I20260808 02:16:46.505950 14477 tablet_service.cc:1467] Processing CreateTablet for tablet 5df2e76ee97842b2b49e6b67187e9559 (DEFAULT_TABLE table=test-table [id=80be4b8a70ac4a35afbcaf2ccab31576]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:46.506273 14477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5df2e76ee97842b2b49e6b67187e9559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:46.506532 14682 tablet_service.cc:1467] Processing CreateTablet for tablet 5df2e76ee97842b2b49e6b67187e9559 (DEFAULT_TABLE table=test-table [id=80be4b8a70ac4a35afbcaf2ccab31576]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:46.506767 14682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5df2e76ee97842b2b49e6b67187e9559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:46.509212 14988 tablet_bootstrap.cc:492] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: Bootstrap starting.
I20260808 02:16:46.509572 14991 tablet_bootstrap.cc:492] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c: Bootstrap starting.
I20260808 02:16:46.509814 14990 tablet_bootstrap.cc:492] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: Bootstrap starting.
I20260808 02:16:46.509861 14988 tablet_bootstrap.cc:654] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:46.510171 14988 log.cc:824] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:46.510277 14991 tablet_bootstrap.cc:654] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:46.510464 14990 tablet_bootstrap.cc:654] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:46.510630 14991 log.cc:824] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:46.510757 14990 log.cc:824] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:46.511492 14990 tablet_bootstrap.cc:492] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: No bootstrap required, opened a new log
I20260808 02:16:46.511500 14991 tablet_bootstrap.cc:492] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c: No bootstrap required, opened a new log
I20260808 02:16:46.511567 14991 ts_tablet_manager.cc:1397] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:46.511567 14990 ts_tablet_manager.cc:1397] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:46.514055 14990 raft_consensus.cc:348] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 } } peers { permanent_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37559 } } peers { permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 } }
I20260808 02:16:46.514055 14991 raft_consensus.cc:348] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 } } peers { permanent_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37559 } } peers { permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 } }
I20260808 02:16:46.514228 14990 raft_consensus.cc:374] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:46.514228 14991 raft_consensus.cc:374] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:46.514272 14991 raft_consensus.cc:729] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3159ef6791a74de0be98a7f6e9ad7a2c, State: Initialized, Role: FOLLOWER
I20260808 02:16:46.514272 14990 raft_consensus.cc:729] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25fb7a89ec964aea910137969cea3e26, State: Initialized, Role: FOLLOWER
I20260808 02:16:46.514400 14990 consensus_queue.cc:260] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 } } peers { permanent_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37559 } } peers { permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 } }
I20260808 02:16:46.514415 14991 consensus_queue.cc:260] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 } } peers { permanent_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37559 } } peers { permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 } }
I20260808 02:16:46.514672 14990 ts_tablet_manager.cc:1428] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:46.514672 14991 ts_tablet_manager.cc:1428] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:46.514712 14580 heartbeater.cc:503] Master 127.19.58.254:42159 was elected leader, sending a full tablet report...
I20260808 02:16:46.514786 14970 heartbeater.cc:503] Master 127.19.58.254:42159 was elected leader, sending a full tablet report...
I20260808 02:16:46.515102 14988 tablet_bootstrap.cc:492] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: No bootstrap required, opened a new log
I20260808 02:16:46.515167 14988 ts_tablet_manager.cc:1397] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260808 02:16:46.517095 14988 raft_consensus.cc:348] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 } } peers { permanent_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37559 } } peers { permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 } }
I20260808 02:16:46.517222 14988 raft_consensus.cc:374] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:46.517242 14988 raft_consensus.cc:729] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcce1431ce2c4036b9f3cdc64f80d475, State: Initialized, Role: FOLLOWER
I20260808 02:16:46.517331 14988 consensus_queue.cc:260] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 } } peers { permanent_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37559 } } peers { permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 } }
I20260808 02:16:46.517550 14988 ts_tablet_manager.cc:1428] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:16:46.517606 14772 heartbeater.cc:503] Master 127.19.58.254:42159 was elected leader, sending a full tablet report...
W20260808 02:16:46.612092 14773 tablet.cc:2369] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:46.679035 14998 raft_consensus.cc:482] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:46.679176 14998 raft_consensus.cc:504] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 } } peers { permanent_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37559 } } peers { permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 } }
I20260808 02:16:46.679548 14998 leader_election.cc:288] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25fb7a89ec964aea910137969cea3e26 (127.19.58.195:39889), fcce1431ce2c4036b9f3cdc64f80d475 (127.19.58.194:35133)
I20260808 02:16:46.683274 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.679825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:50919 (local address 127.19.58.194:35133)
0808 02:16:46.679873 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.679890 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.679903 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:16:46.679965 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.679966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.679977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.680032 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:46.680197 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.680955 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.682377 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.682936 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.683005 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.683013 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.683164 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.683201 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 02:16:46.683274 14984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.679863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:40525 (local address 127.19.58.195:39889)
0808 02:16:46.679909 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.679910 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.680065 (+ 155us) server_negotiation.cc:407] Connection header received
0808 02:16:46.680102 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.680103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.680110 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.680150 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:46.680289 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.681004 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.682382 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.682932 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.683005 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.683013 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.683164 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.683201 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 02:16:46.683288 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.679779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:35133 (local address 127.19.58.193:50919)
0808 02:16:46.679831 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.679844 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:46.679894 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:46.680052 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:46.680053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:46.680054 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:46.680131 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.680133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.681030 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.681031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.682286 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.682291 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.683012 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.683013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.683094 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.683098 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:46.683154 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 02:16:46.683199 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":27}
I20260808 02:16:46.683296 15000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.679822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:39889 (local address 127.19.58.193:40525)
0808 02:16:46.679992 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.679999 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:46.680052 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:46.680170 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:46.680171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:46.680172 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:46.680231 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.680232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.681069 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.681069 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.682288 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:46.682291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.683014 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.683014 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:46.683103 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.683106 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:46.683154 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 02:16:46.683196 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":5,"thread_start_us":114,"threads_started":1}
I20260808 02:16:46.683689 14713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df2e76ee97842b2b49e6b67187e9559" candidate_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" is_pre_election: true
I20260808 02:16:46.683884 14713 raft_consensus.cc:2454] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3159ef6791a74de0be98a7f6e9ad7a2c in term 0.
I20260808 02:16:46.683898 14901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df2e76ee97842b2b49e6b67187e9559" candidate_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fb7a89ec964aea910137969cea3e26" is_pre_election: true
I20260808 02:16:46.684044 14901 raft_consensus.cc:2454] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3159ef6791a74de0be98a7f6e9ad7a2c in term 0.
I20260808 02:16:46.684135 14413 leader_election.cc:302] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3159ef6791a74de0be98a7f6e9ad7a2c, fcce1431ce2c4036b9f3cdc64f80d475; no voters:
I20260808 02:16:46.684305 14998 raft_consensus.cc:2781] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:16:46.684360 14998 raft_consensus.cc:482] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:16:46.684376 14998 raft_consensus.cc:3037] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:46.685132 14998 raft_consensus.cc:504] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 } } peers { permanent_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37559 } } peers { permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 } }
I20260808 02:16:46.685267 14998 leader_election.cc:288] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [CANDIDATE]: Term 1 election: Requested vote from peers 25fb7a89ec964aea910137969cea3e26 (127.19.58.195:39889), fcce1431ce2c4036b9f3cdc64f80d475 (127.19.58.194:35133)
I20260808 02:16:46.685442 14901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df2e76ee97842b2b49e6b67187e9559" candidate_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25fb7a89ec964aea910137969cea3e26"
I20260808 02:16:46.685479 14713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5df2e76ee97842b2b49e6b67187e9559" candidate_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcce1431ce2c4036b9f3cdc64f80d475"
I20260808 02:16:46.685514 14901 raft_consensus.cc:3037] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:46.685544 14713 raft_consensus.cc:3037] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:46.686209 14901 raft_consensus.cc:2454] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3159ef6791a74de0be98a7f6e9ad7a2c in term 1.
I20260808 02:16:46.686239 14713 raft_consensus.cc:2454] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3159ef6791a74de0be98a7f6e9ad7a2c in term 1.
I20260808 02:16:46.686378 14413 leader_election.cc:302] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3159ef6791a74de0be98a7f6e9ad7a2c, fcce1431ce2c4036b9f3cdc64f80d475; no voters:
I20260808 02:16:46.686482 14998 raft_consensus.cc:2781] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:46.686621 14998 raft_consensus.cc:686] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 1 LEADER]: Becoming Leader. State: Replica: 3159ef6791a74de0be98a7f6e9ad7a2c, State: Running, Role: LEADER
I20260808 02:16:46.686708 14998 consensus_queue.cc:237] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 } } peers { permanent_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37559 } } peers { permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 } }
I20260808 02:16:46.687521 14297 catalog_manager.cc:5495] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3159ef6791a74de0be98a7f6e9ad7a2c (127.19.58.193). New cstate: current_term: 1 leader_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3159ef6791a74de0be98a7f6e9ad7a2c" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 37559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 } health_report { overall_health: UNKNOWN } } }
W20260808 02:16:46.731906 14581 tablet.cc:2369] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:46.733301 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.730988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49880 (local address 127.19.58.193:37559)
0808 02:16:46.731046 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:46.731048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:46.731055 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:16:46.731096 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:46.731096 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:46.731116 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:46.731180 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:46.731304 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.731807 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.732580 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:46.732773 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:46.732829 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:46.732975 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:46.733064 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:46.733147 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:46.733196 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 02:16:46.733206 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 02:16:46.734059 14974 tablet.cc:2369] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:46.735014 14901 raft_consensus.cc:1261] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26 [term 1 FOLLOWER]: Refusing update from remote peer 3159ef6791a74de0be98a7f6e9ad7a2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:46.735015 14713 raft_consensus.cc:1261] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475 [term 1 FOLLOWER]: Refusing update from remote peer 3159ef6791a74de0be98a7f6e9ad7a2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:46.735339 15002 consensus_queue.cc:1035] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcce1431ce2c4036b9f3cdc64f80d475" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 35133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:46.735468 14998 consensus_queue.cc:1035] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "25fb7a89ec964aea910137969cea3e26" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 39889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:46.738021 15005 mvcc.cc:205] Tried to move back new op lower bound from 7316092545984106496 to 7316092545789288448. Current Snapshot: MvccSnapshot[applied={T|T < 7316092545984106496}]
I20260808 02:16:46.738035 15007 mvcc.cc:205] Tried to move back new op lower bound from 7316092545984106496 to 7316092545789288448. Current Snapshot: MvccSnapshot[applied={T|T < 7316092545984106496}]
I20260808 02:16:46.738142 15008 mvcc.cc:205] Tried to move back new op lower bound from 7316092545984106496 to 7316092545789288448. Current Snapshot: MvccSnapshot[applied={T|T < 7316092545984106496}]
I20260808 02:16:46.740737 14297 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:46.740833 14297 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:46.741554 14297 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b306dbdd8caa49b8b40e7590e8ded6a7: Sending DeleteTablet for 3 replicas of tablet 5df2e76ee97842b2b49e6b67187e9559
I20260808 02:16:46.741892 14682 tablet_service.cc:1514] Processing DeleteTablet for tablet 5df2e76ee97842b2b49e6b67187e9559 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:46 UTC) from {username='slave'} at 127.0.0.1:38926
I20260808 02:16:46.741928 14877 tablet_service.cc:1514] Processing DeleteTablet for tablet 5df2e76ee97842b2b49e6b67187e9559 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:46 UTC) from {username='slave'} at 127.0.0.1:33962
I20260808 02:16:46.742094 14477 tablet_service.cc:1514] Processing DeleteTablet for tablet 5df2e76ee97842b2b49e6b67187e9559 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:46 UTC) from {username='slave'} at 127.0.0.1:49878
I20260808 02:16:46.742141 15018 tablet_replica.cc:330] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: stopping tablet replica
I20260808 02:16:46.742311 15018 raft_consensus.cc:2229] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:46.742349 15019 tablet_replica.cc:330] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c: stopping tablet replica
I20260808 02:16:46.742445 15019 raft_consensus.cc:2229] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:16:46.742558 15019 raft_consensus.cc:2258] T 5df2e76ee97842b2b49e6b67187e9559 P 3159ef6791a74de0be98a7f6e9ad7a2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:46.742558 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 14390
I20260808 02:16:46.746481 15017 tablet_replica.cc:330] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: stopping tablet replica
I20260808 02:16:46.746570 15017 raft_consensus.cc:2229] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:46.746654 15017 raft_consensus.cc:2258] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:46.748220 15017 ts_tablet_manager.cc:1905] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:46.748359 15018 raft_consensus.cc:2258] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:46.748795 15018 ts_tablet_manager.cc:1905] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:46.749943 15017 ts_tablet_manager.cc:1918] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:46.750018 15017 log.cc:1196] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/5df2e76ee97842b2b49e6b67187e9559
I20260808 02:16:46.750262 15017 ts_tablet_manager.cc:1939] T 5df2e76ee97842b2b49e6b67187e9559 P fcce1431ce2c4036b9f3cdc64f80d475: Deleting consensus metadata
I20260808 02:16:46.750535 15018 ts_tablet_manager.cc:1918] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:46.750608 15018 log.cc:1196] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/5df2e76ee97842b2b49e6b67187e9559
I20260808 02:16:46.750738 14277 catalog_manager.cc:4841] TS fcce1431ce2c4036b9f3cdc64f80d475 (127.19.58.194:35133): tablet 5df2e76ee97842b2b49e6b67187e9559 (table test-table [id=80be4b8a70ac4a35afbcaf2ccab31576]) successfully deleted
I20260808 02:16:46.750845 15018 ts_tablet_manager.cc:1939] T 5df2e76ee97842b2b49e6b67187e9559 P 25fb7a89ec964aea910137969cea3e26: Deleting consensus metadata
I20260808 02:16:46.751276 14279 catalog_manager.cc:4841] TS 25fb7a89ec964aea910137969cea3e26 (127.19.58.195:39889): tablet 5df2e76ee97842b2b49e6b67187e9559 (table test-table [id=80be4b8a70ac4a35afbcaf2ccab31576]) successfully deleted
W20260808 02:16:46.752000 14279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:37559 (error 108)
I20260808 02:16:46.752242 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 14585
I20260808 02:16:46.754323 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:46.752318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:37559 (local address 127.0.0.1:49896)
0808 02:16:46.754238 (+ 1920us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:46.754290 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:37559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1893}
W20260808 02:16:46.756505 14279 catalog_manager.cc:4584] TS 3159ef6791a74de0be98a7f6e9ad7a2c (127.19.58.193:37559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5df2e76ee97842b2b49e6b67187e9559: Network error: Client connection negotiation failed: client connection to 127.19.58.193:37559: connect: Connection refused (error 111)
I20260808 02:16:46.761535 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 14776
I20260808 02:16:46.771283 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 14253
2026-08-08T02:16:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (898 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:46 dist-test-slave-1l3l krb5kdc[15026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:46 dist-test-slave-1l3l krb5kdc[15026](info): set up 2 sockets
Aug 08 02:16:46 dist-test-slave-1l3l krb5kdc[15026](info): commencing operation
krb5kdc: starting...
W20260808 02:16:47.898300 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:16:47 dist-test-slave-1l3l krb5kdc[15026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:16:48.014336 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:34103
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:34103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:48.095986 15049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:48.096210 15049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:48.096230 15049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:48.097317 15049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:48.097352 15049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:48.097364 15049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:48.097375 15049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:48.097386 15049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:48.098807 15049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:34103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:34103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:48.098996 15049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:48.099164 15049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:48.103650 15055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:48.103652 15057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:48.103778 15049 server_base.cc:1034] running on GCE node
W20260808 02:16:48.103662 15054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:48.104041 15049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:48.104274 15049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:48.105406 15049 hybrid_clock.cc:648] HybridClock initialized: now 1786155408105391 us; error 29 us; skew 500 ppm
Aug 08 02:16:48 dist-test-slave-1l3l krb5kdc[15026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155408, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:48.110705 15049 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:16:48.111140 15049 webserver.cc:460] Webserver started at http://127.19.58.254:34433/ using document root <none> and password file <none>
I20260808 02:16:48.111366 15049 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:48.111419 15049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:48.111549 15049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:48.112684 15049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "23684a56ca7c417d9f1265b9a04594ef"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "ba193504c1776e74bce4ee0c9c29c3ab"
server_key_iv: "dba49d9e9d17fafc97df6f40ae399fc2"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.113045 15049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "23684a56ca7c417d9f1265b9a04594ef"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "ba193504c1776e74bce4ee0c9c29c3ab"
server_key_iv: "dba49d9e9d17fafc97df6f40ae399fc2"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.114446 15049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:16:48.115237 15067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:48.115424 15049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:48.115502 15049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "23684a56ca7c417d9f1265b9a04594ef"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "ba193504c1776e74bce4ee0c9c29c3ab"
server_key_iv: "dba49d9e9d17fafc97df6f40ae399fc2"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.115576 15049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:48.137828 15049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:48.139994 15049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:48.140141 15049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:48.143909 15049 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:34103
I20260808 02:16:48.143955 15132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:34103 every 8 connection(s)
I20260808 02:16:48.144271 15049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:48.145112 15134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:48.147400 15134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef: Bootstrap starting.
I20260808 02:16:48.147995 15134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:48.148255 15134 log.cc:824] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:48.148988 15134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef: No bootstrap required, opened a new log
I20260808 02:16:48.149158 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 15049
I20260808 02:16:48.149260 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:48.149437 19691 external_mini_cluster.cc:1428] Setting key 90331f2eeb5d445e96cec426b603e981
I20260808 02:16:48.151125 15134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23684a56ca7c417d9f1265b9a04594ef" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34103 } }
I20260808 02:16:48.151298 15134 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:48.151353 15134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23684a56ca7c417d9f1265b9a04594ef, State: Initialized, Role: FOLLOWER
I20260808 02:16:48.151473 15134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23684a56ca7c417d9f1265b9a04594ef" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34103 } }
I20260808 02:16:48.151548 15134 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:48.151577 15134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:48.151623 15134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:48.152781 15134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23684a56ca7c417d9f1265b9a04594ef" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34103 } }
I20260808 02:16:48.152899 15134 leader_election.cc:302] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23684a56ca7c417d9f1265b9a04594ef; no voters:
I20260808 02:16:48.153059 15134 leader_election.cc:288] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:48.153110 15142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:48.153282 15142 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [term 1 LEADER]: Becoming Leader. State: Replica: 23684a56ca7c417d9f1265b9a04594ef, State: Running, Role: LEADER
I20260808 02:16:48.153324 15134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:48.153390 15142 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23684a56ca7c417d9f1265b9a04594ef" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34103 } }
I20260808 02:16:48.153762 15143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23684a56ca7c417d9f1265b9a04594ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23684a56ca7c417d9f1265b9a04594ef" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34103 } } }
I20260808 02:16:48.153905 15143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:48.153759 15144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23684a56ca7c417d9f1265b9a04594ef. Latest consensus state: current_term: 1 leader_uuid: "23684a56ca7c417d9f1265b9a04594ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23684a56ca7c417d9f1265b9a04594ef" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34103 } } }
I20260808 02:16:48.153968 15144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef [sys.catalog]: This master's current role is: LEADER
W20260808 02:16:48.155508 15163 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:16:48.155566 15163 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:16:48.155774 15153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:48.156059 15153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 02:16:48 dist-test-slave-1l3l krb5kdc[15026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:48.157814 15153 catalog_manager.cc:1347] Generated new cluster ID: e989d815dd454b6ca2ec1bf1b31960f2
I20260808 02:16:48.157872 15153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:48.161541 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.149940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45832 (local address 127.19.58.254:34103)
0808 02:16:48.150159 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.150169 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.150204 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:16:48.151087 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.151097 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.151164 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.151317 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:48.152100 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.153528 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.154611 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.155239 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.155353 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.158168 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:48.158172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:48.158172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:48.158183 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:48.159769 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:48.160087 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:48.160090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:48.160091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:48.160202 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:48.160390 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:48.160391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:48.160392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:48.160613 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:48.161235 (+ 622us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.161291 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.161400 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
I20260808 02:16:48.173188 15153 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:48.173853 15153 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:48.186986 15153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef: Generated new TSK 0
I20260808 02:16:48.187160 15153 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.58.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
Entry for principal kudu/127.19.58.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab.
I20260808 02:16:48.214154 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:34103
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:48.294325 15169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:48.294520 15169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:48.294548 15169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:48.295662 15169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:48.295727 15169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:48.295753 15169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:16:48.297142 15169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.193.keytab
--principal=kudu/127.19.58.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:34103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:48.297317 15169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:48.297482 15169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:48.302136 15178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:48.302142 15181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:48.302155 15179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:48.302316 15169 server_base.cc:1034] running on GCE node
I20260808 02:16:48.302527 15169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:48.302719 15169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:48.303864 15169 hybrid_clock.cc:648] HybridClock initialized: now 1786155408303843 us; error 38 us; skew 500 ppm
Aug 08 02:16:48 dist-test-slave-1l3l krb5kdc[15026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155408, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:48.306252 15169 init.cc:377] Logged in from keytab as kudu/127.19.58.193@KRBTEST.COM (short username kudu)
I20260808 02:16:48.306650 15169 webserver.cc:460] Webserver started at http://127.19.58.193:39175/ using document root <none> and password file <none>
I20260808 02:16:48.306816 15169 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:48.306854 15169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:48.306941 15169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:48.307914 15169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "75639ad976e3488ea4bcb11509760102"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "9020010b49b81fc6dadf690f74b6c332"
server_key_iv: "b64f2499fbb40caab0d4d91d69594c3e"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.308197 15169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "75639ad976e3488ea4bcb11509760102"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "9020010b49b81fc6dadf690f74b6c332"
server_key_iv: "b64f2499fbb40caab0d4d91d69594c3e"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.309446 15169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:48.310057 15195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:48.310205 15169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:48.310290 15169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "75639ad976e3488ea4bcb11509760102"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "9020010b49b81fc6dadf690f74b6c332"
server_key_iv: "b64f2499fbb40caab0d4d91d69594c3e"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.310348 15169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:48.326303 15169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:48.328403 15169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:48.328537 15169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:48.328749 15169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:48.329046 15169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:48.329080 15169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:48.329108 15169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:48.329126 15169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:48.334519 15169 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:34471
I20260808 02:16:48.334578 15358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:34471 every 8 connection(s)
I20260808 02:16:48.334848 15169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:48.339104 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 15169
I20260808 02:16:48.339202 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:16:48.339421 19691 external_mini_cluster.cc:1428] Setting key ba0a2b21639235ecf0f543255e9ce918
Aug 08 02:16:48 dist-test-slave-1l3l krb5kdc[15026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155408, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.193@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:48.343484 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.335492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:37997 (local address 127.19.58.254:34103)
0808 02:16:48.335554 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.335556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.335765 (+ 209us) server_negotiation.cc:407] Connection header received
0808 02:16:48.336235 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.336236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.336251 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.336313 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:48.337161 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.337659 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.338573 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.338779 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.338845 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.340942 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:48.340944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:48.340944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:48.340954 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:48.342210 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:48.342574 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:48.342575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:48.342575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:48.342626 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:48.342853 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:48.342854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:48.342854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:48.342965 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:48.343022 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.343299 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.343392 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 02:16:48.343623 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.335454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:34103 (local address 127.19.58.193:37997)
0808 02:16:48.335608 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.335649 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:48.336144 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:48.336360 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:48.336362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:48.336886 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:48.337066 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.337070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.337728 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.337729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.338498 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.338500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.338851 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.338852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.338942 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.339380 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:48.339388 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:48.340851 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:48.342286 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:48.342293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:48.342295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:48.342508 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:48.342698 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:48.342699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:48.342699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:48.342798 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:48.343022 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:48.343030 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:48.343163 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 02:16:48.343267 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 02:16:48.343422 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260808 02:16:48.344108 15359 heartbeater.cc:347] Connected to a master server at 127.19.58.254:34103
I20260808 02:16:48.344187 15359 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:48.344339 15359 heartbeater.cc:511] Master 127.19.58.254:34103 requested a full tablet report, sending...
I20260808 02:16:48.344861 15086 ts_manager.cc:194] Registered new tserver with Master: 75639ad976e3488ea4bcb11509760102 (127.19.58.193:34471)
I20260808 02:16:48.345729 15086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.193@KRBTEST.COM'} at 127.19.58.193:37997
WARNING: no policy specified for kudu/127.19.58.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
Entry for principal kudu/127.19.58.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab.
I20260808 02:16:48.364421 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:34103
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:48.446158 15364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:48.446365 15364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:48.446388 15364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:48.447464 15364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:48.447497 15364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:48.447521 15364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:16:48.448941 15364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.194.keytab
--principal=kudu/127.19.58.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:34103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:48.449128 15364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:48.449309 15364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:48.454023 15373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:48.454046 15374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:48.454023 15381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:48.454388 15364 server_base.cc:1034] running on GCE node
I20260808 02:16:48.454586 15364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:48.454790 15364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:48.455922 15364 hybrid_clock.cc:648] HybridClock initialized: now 1786155408455907 us; error 33 us; skew 500 ppm
Aug 08 02:16:48 dist-test-slave-1l3l krb5kdc[15026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155408, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:48.458297 15364 init.cc:377] Logged in from keytab as kudu/127.19.58.194@KRBTEST.COM (short username kudu)
I20260808 02:16:48.458803 15364 webserver.cc:460] Webserver started at http://127.19.58.194:43605/ using document root <none> and password file <none>
I20260808 02:16:48.459015 15364 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:48.459064 15364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:48.459187 15364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:48.460218 15364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "872eaabce5c5475bb867d7bdb90b8b77"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "17beb9a6e3954701fe8a1c1819c2fae0"
server_key_iv: "8944188449f7211a1e28c495a48aa334"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.460556 15364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "872eaabce5c5475bb867d7bdb90b8b77"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "17beb9a6e3954701fe8a1c1819c2fae0"
server_key_iv: "8944188449f7211a1e28c495a48aa334"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.461760 15364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:16:48.462424 15393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:48.462576 15364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:48.462661 15364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "872eaabce5c5475bb867d7bdb90b8b77"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "17beb9a6e3954701fe8a1c1819c2fae0"
server_key_iv: "8944188449f7211a1e28c495a48aa334"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.462724 15364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:48.480293 15364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:48.482465 15364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:48.482606 15364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:48.482829 15364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:48.483137 15364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:48.483172 15364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:48.483202 15364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:48.483220 15364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:48.488686 15364 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:46119
I20260808 02:16:48.488795 15559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:46119 every 8 connection(s)
I20260808 02:16:48.489051 15364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:16:48.489264 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 15364
I20260808 02:16:48.489377 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:16:48.489583 19691 external_mini_cluster.cc:1428] Setting key 3d94938cc9bf6d2bd4a0363233e8d0ca
Aug 08 02:16:48 dist-test-slave-1l3l krb5kdc[15026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155408, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.194@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.58.195@KRBTEST.COM; defaulting to no policy
I20260808 02:16:48.498932 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.489829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:40395 (local address 127.19.58.254:34103)
0808 02:16:48.489908 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.489911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.490112 (+ 201us) server_negotiation.cc:407] Connection header received
0808 02:16:48.490689 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.490690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.490702 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.490765 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:48.491647 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.492301 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.493536 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.493806 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.493876 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.495962 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:48.495963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:48.495964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:48.495973 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:48.497448 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:48.497917 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:48.497918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:48.497918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:48.497978 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:48.498244 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:48.498244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:48.498244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:48.498359 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:48.498412 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.498742 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.498849 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 02:16:48.499109 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.489807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:34103 (local address 127.19.58.194:40395)
0808 02:16:48.489979 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.490029 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:48.490579 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:48.490825 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:48.490828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:48.491369 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:48.491555 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.491560 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.492390 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.492392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.493469 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.493472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.493879 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.493880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.493974 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.494454 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:48.494463 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:48.495868 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:48.497545 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:48.497553 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:48.497555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:48.497833 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:48.498041 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:48.498042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:48.498043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:48.498179 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:48.498427 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:48.498434 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:48.498601 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 02:16:48.498717 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 02:16:48.498878 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260808 02:16:48.499560 15560 heartbeater.cc:347] Connected to a master server at 127.19.58.254:34103
I20260808 02:16:48.499647 15560 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:48.499833 15560 heartbeater.cc:511] Master 127.19.58.254:34103 requested a full tablet report, sending...
I20260808 02:16:48.500241 15086 ts_manager.cc:194] Registered new tserver with Master: 872eaabce5c5475bb867d7bdb90b8b77 (127.19.58.194:46119)
I20260808 02:16:48.500588 15086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.194@KRBTEST.COM'} at 127.19.58.194:40395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
Entry for principal kudu/127.19.58.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab.
I20260808 02:16:48.516005 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:34103
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:48.597110 15568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:48.597325 15568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:48.597349 15568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:48.598440 15568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:48.598476 15568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:48.598500 15568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:16:48.599927 15568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.195.keytab
--principal=kudu/127.19.58.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:34103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:48.600123 15568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:48.600307 15568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:48.605158 15577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:48.605191 15575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:48.605160 15574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:48.605175 15568 server_base.cc:1034] running on GCE node
I20260808 02:16:48.605504 15568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:48.605726 15568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:48.606874 15568 hybrid_clock.cc:648] HybridClock initialized: now 1786155408606861 us; error 25 us; skew 500 ppm
Aug 08 02:16:48 dist-test-slave-1l3l krb5kdc[15026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155408, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:48.609376 15568 init.cc:377] Logged in from keytab as kudu/127.19.58.195@KRBTEST.COM (short username kudu)
I20260808 02:16:48.609776 15568 webserver.cc:460] Webserver started at http://127.19.58.195:40169/ using document root <none> and password file <none>
I20260808 02:16:48.609992 15568 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:48.610044 15568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:48.610172 15568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:48.611200 15568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "979e7f11ef774efd8c97338516687977"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "313663e2d179ecdf75590265048a4ce4"
server_key_iv: "5bb75764b5d4a9fd98f8d556a33ad247"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.611527 15568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "979e7f11ef774efd8c97338516687977"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "313663e2d179ecdf75590265048a4ce4"
server_key_iv: "5bb75764b5d4a9fd98f8d556a33ad247"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.612819 15568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 02:16:48.613525 15585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:48.613672 15568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:48.613765 15568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "979e7f11ef774efd8c97338516687977"
format_stamp: "Formatted at 2026-08-08 02:16:48 on dist-test-slave-1l3l"
server_key: "313663e2d179ecdf75590265048a4ce4"
server_key_iv: "5bb75764b5d4a9fd98f8d556a33ad247"
server_key_version: "encryptionkey@0"
I20260808 02:16:48.613847 15568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:48.624898 15568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:48.626458 15568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:48.626583 15568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:48.626789 15568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:48.627065 15568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:48.627099 15568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:48.627126 15568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:48.627151 15568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:48.632892 15568 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:42771
I20260808 02:16:48.632931 15747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:42771 every 8 connection(s)
I20260808 02:16:48.633222 15568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
Aug 08 02:16:48 dist-test-slave-1l3l krb5kdc[15026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155408, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.195@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:16:48.641304 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 15568
I20260808 02:16:48.641412 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:16:48.641582 19691 external_mini_cluster.cc:1428] Setting key 1b1c49c8fb53c6f55f73284f2ea066ce
I20260808 02:16:48.642323 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.633914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:51417 (local address 127.19.58.254:34103)
0808 02:16:48.633975 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.633977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.634155 (+ 178us) server_negotiation.cc:407] Connection header received
0808 02:16:48.634620 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.634621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.634633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.634710 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:48.635661 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.636193 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.637258 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.637489 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.637551 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.639839 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:48.639840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:48.639840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:48.639848 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:48.640953 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:48.641350 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:48.641351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:48.641351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:48.641411 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:48.641634 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:48.641635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:48.641635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:48.641761 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:48.641831 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.642153 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.642239 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:16:48.642490 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.633864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:34103 (local address 127.19.58.195:51417)
0808 02:16:48.634034 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.634076 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:48.634536 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:48.634766 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:48.634769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:48.635393 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:48.635587 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.635592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.636281 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.636282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.637133 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.637136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.637597 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.637598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.637717 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.638258 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:16:48.638266 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:48.639681 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:48.641028 (+ 1347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:48.641035 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:48.641038 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:48.641278 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:48.641477 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:16:48.641478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:16:48.641478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:16:48.641575 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:16:48.641834 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:48.641842 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:48.642002 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 02:16:48.642111 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 02:16:48.642280 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260808 02:16:48.643074 15748 heartbeater.cc:347] Connected to a master server at 127.19.58.254:34103
I20260808 02:16:48.643164 15748 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:48.643327 15748 heartbeater.cc:511] Master 127.19.58.254:34103 requested a full tablet report, sending...
I20260808 02:16:48.643653 15086 ts_manager.cc:194] Registered new tserver with Master: 979e7f11ef774efd8c97338516687977 (127.19.58.195:42771)
I20260808 02:16:48.644037 15086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.58.195@KRBTEST.COM'} at 127.19.58.195:51417
I20260808 02:16:48.653398 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:16:48.660642 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.654848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43580 (local address 127.19.58.254:34103)
0808 02:16:48.654895 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.654897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.655016 (+ 119us) server_negotiation.cc:407] Connection header received
0808 02:16:48.655063 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.655064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.655074 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.655125 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:48.655737 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.656218 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.657007 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.657190 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.657244 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.658448 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:48.658449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:48.658449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:48.658455 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:48.659770 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:48.660072 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:48.660072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:48.660073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:48.660119 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:48.660248 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:48.660249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:48.660249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:48.660374 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:48.660444 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.660527 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.660580 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 02:16:48.662042 15086 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:16:48.663555 15086 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:16:48.669553 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.665766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50878 (local address 127.19.58.195:42771)
0808 02:16:48.666056 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.666061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.666070 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:16:48.666109 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.666112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.666126 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.666203 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:48.666356 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.667140 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.668461 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.669164 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.669236 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.669289 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.669433 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.669474 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":90,"threads_started":1}
I20260808 02:16:48.669545 15765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.665609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:42771 (local address 127.0.0.1:50878)
0808 02:16:48.665864 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.665905 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:48.665973 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:48.666198 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:48.666199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:48.666200 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:48.666253 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.666255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.667227 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.667228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.668366 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.668369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.669248 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.669249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.669328 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.669344 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 02:16:48.669413 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:16:48.669470 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"spinlock_wait_cycles":768,"thread_start_us":53,"threads_started":1}
I20260808 02:16:48.669732 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.665851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.19.58.194:46119)
0808 02:16:48.666365 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.666369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.666378 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:16:48.666414 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.666416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.666432 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.666500 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:48.666698 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.667747 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.668759 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.669254 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.669332 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.669388 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.669599 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.669645 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":83,"threads_started":1}
I20260808 02:16:48.669884 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.665615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:46119 (local address 127.0.0.1:56468)
0808 02:16:48.666070 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.666079 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:48.666114 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:48.666580 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:48.666581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:48.666583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:48.666617 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.666619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.667847 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.667848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.668685 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.668687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.669422 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.669423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.669476 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.669479 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:48.669524 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 02:16:48.669558 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":67,"threads_started":1}
I20260808 02:16:48.669974 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.665610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:34471 (local address 127.0.0.1:42420)
0808 02:16:48.665938 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.665947 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:48.665999 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:48.666181 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:48.666183 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:48.666185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:48.666247 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.666248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.667637 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.667639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.668724 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.668727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.669651 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.669652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.669721 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.669725 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:48.669861 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 02:16:48.669904 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":22,"thread_start_us":55,"threads_started":1}
I20260808 02:16:48.670259 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.665783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42420 (local address 127.19.58.193:34471)
0808 02:16:48.666001 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.666006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.666046 (+ 40us) server_negotiation.cc:407] Connection header received
0808 02:16:48.666085 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.666087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.666102 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.666166 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:48.666417 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.667527 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.668814 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.669500 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.669561 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.669613 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.670140 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.670180 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":100,"threads_started":1}
I20260808 02:16:48.670447 15660 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd29b59fddf4a66841105fada3d10cd (DEFAULT_TABLE table=test-table [id=935e4b452d994f49bc9004239bdcf9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:48.670598 15456 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd29b59fddf4a66841105fada3d10cd (DEFAULT_TABLE table=test-table [id=935e4b452d994f49bc9004239bdcf9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:48.670862 15660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd29b59fddf4a66841105fada3d10cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:48.670887 15456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd29b59fddf4a66841105fada3d10cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:48.670863 15265 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd29b59fddf4a66841105fada3d10cd (DEFAULT_TABLE table=test-table [id=935e4b452d994f49bc9004239bdcf9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:48.671182 15265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd29b59fddf4a66841105fada3d10cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:48.673856 15771 tablet_bootstrap.cc:492] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102: Bootstrap starting.
I20260808 02:16:48.674407 15773 tablet_bootstrap.cc:492] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: Bootstrap starting.
I20260808 02:16:48.674629 15771 tablet_bootstrap.cc:654] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:48.675079 15773 tablet_bootstrap.cc:654] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:48.675218 15771 log.cc:824] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:48.675638 15773 log.cc:824] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:48.675954 15774 tablet_bootstrap.cc:492] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: Bootstrap starting.
I20260808 02:16:48.676100 15771 tablet_bootstrap.cc:492] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102: No bootstrap required, opened a new log
I20260808 02:16:48.676167 15771 ts_tablet_manager.cc:1397] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:48.676375 15773 tablet_bootstrap.cc:492] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: No bootstrap required, opened a new log
I20260808 02:16:48.676440 15773 ts_tablet_manager.cc:1397] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:48.676656 15774 tablet_bootstrap.cc:654] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:48.677222 15774 log.cc:824] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:48.678014 15774 tablet_bootstrap.cc:492] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: No bootstrap required, opened a new log
I20260808 02:16:48.678085 15774 ts_tablet_manager.cc:1397] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:48.678051 15773 raft_consensus.cc:348] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75639ad976e3488ea4bcb11509760102" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34471 } } peers { permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 } } peers { permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 } }
I20260808 02:16:48.678118 15771 raft_consensus.cc:348] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75639ad976e3488ea4bcb11509760102" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34471 } } peers { permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 } } peers { permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 } }
I20260808 02:16:48.678234 15773 raft_consensus.cc:374] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:48.678282 15771 raft_consensus.cc:374] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:48.678278 15773 raft_consensus.cc:729] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 979e7f11ef774efd8c97338516687977, State: Initialized, Role: FOLLOWER
I20260808 02:16:48.678315 15771 raft_consensus.cc:729] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75639ad976e3488ea4bcb11509760102, State: Initialized, Role: FOLLOWER
I20260808 02:16:48.678422 15773 consensus_queue.cc:260] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75639ad976e3488ea4bcb11509760102" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34471 } } peers { permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 } } peers { permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 } }
I20260808 02:16:48.678457 15771 consensus_queue.cc:260] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75639ad976e3488ea4bcb11509760102" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34471 } } peers { permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 } } peers { permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 } }
I20260808 02:16:48.678694 15773 ts_tablet_manager.cc:1428] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 02:16:48.678726 15771 ts_tablet_manager.cc:1428] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:48.678818 15748 heartbeater.cc:503] Master 127.19.58.254:34103 was elected leader, sending a full tablet report...
I20260808 02:16:48.678817 15359 heartbeater.cc:503] Master 127.19.58.254:34103 was elected leader, sending a full tablet report...
I20260808 02:16:48.680195 15774 raft_consensus.cc:348] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75639ad976e3488ea4bcb11509760102" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34471 } } peers { permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 } } peers { permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 } }
I20260808 02:16:48.680358 15774 raft_consensus.cc:374] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:48.680400 15774 raft_consensus.cc:729] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872eaabce5c5475bb867d7bdb90b8b77, State: Initialized, Role: FOLLOWER
I20260808 02:16:48.680477 15774 consensus_queue.cc:260] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75639ad976e3488ea4bcb11509760102" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34471 } } peers { permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 } } peers { permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 } }
I20260808 02:16:48.680665 15774 ts_tablet_manager.cc:1428] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:48.680732 15560 heartbeater.cc:503] Master 127.19.58.254:34103 was elected leader, sending a full tablet report...
W20260808 02:16:48.739766 15561 tablet.cc:2369] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 02:16:48.835584 15360 tablet.cc:2369] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:48.856657 15781 raft_consensus.cc:482] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:48.856801 15781 raft_consensus.cc:504] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75639ad976e3488ea4bcb11509760102" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34471 } } peers { permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 } } peers { permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 } }
I20260808 02:16:48.857163 15781 leader_election.cc:288] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 979e7f11ef774efd8c97338516687977 (127.19.58.195:42771), 872eaabce5c5475bb867d7bdb90b8b77 (127.19.58.194:46119)
I20260808 02:16:48.860242 15766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.857416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:57493 (local address 127.19.58.195:42771)
0808 02:16:48.857481 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.857483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.857570 (+ 87us) server_negotiation.cc:407] Connection header received
0808 02:16:48.857636 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.857637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.857645 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.857703 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:48.857877 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.858432 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.859502 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.859918 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.859976 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.859985 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.860130 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.860169 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 02:16:48.860244 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.857426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:55781 (local address 127.19.58.194:46119)
0808 02:16:48.857476 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.857478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.857546 (+ 68us) server_negotiation.cc:407] Connection header received
0808 02:16:48.857602 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.857603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.857612 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.857665 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:48.857827 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.858430 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.859514 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.859936 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.860000 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.860010 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.860145 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.860180 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:16:48.860267 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.857347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:46119 (local address 127.19.58.193:55781)
0808 02:16:48.857475 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.857487 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:48.857546 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:48.857676 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:48.857677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:48.857679 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:48.857762 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.857765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.858510 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.858511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.859428 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.859430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.860021 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.860022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.860084 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.860088 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:48.860132 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:16:48.860174 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":42,"thread_start_us":66,"threads_started":1}
I20260808 02:16:48.860257 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.857347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:42771 (local address 127.19.58.193:57493)
0808 02:16:48.857486 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.857497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:48.857577 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:48.857706 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:48.857707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:48.857709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:48.857788 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.857790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.858507 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.858508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.859423 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:48.859426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.859994 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.859995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:48.860069 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.860073 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:48.860117 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:16:48.860160 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":3,"thread_start_us":76,"threads_started":1}
I20260808 02:16:48.860606 15485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd29b59fddf4a66841105fada3d10cd" candidate_uuid: "75639ad976e3488ea4bcb11509760102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872eaabce5c5475bb867d7bdb90b8b77" is_pre_election: true
I20260808 02:16:48.860756 15485 raft_consensus.cc:2454] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75639ad976e3488ea4bcb11509760102 in term 0.
I20260808 02:16:48.861016 15203 leader_election.cc:302] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75639ad976e3488ea4bcb11509760102, 872eaabce5c5475bb867d7bdb90b8b77; no voters:
I20260808 02:16:48.861112 15687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd29b59fddf4a66841105fada3d10cd" candidate_uuid: "75639ad976e3488ea4bcb11509760102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979e7f11ef774efd8c97338516687977" is_pre_election: true
I20260808 02:16:48.861146 15781 raft_consensus.cc:2781] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:16:48.861209 15781 raft_consensus.cc:482] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:16:48.861233 15781 raft_consensus.cc:3037] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:48.861253 15687 raft_consensus.cc:2454] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75639ad976e3488ea4bcb11509760102 in term 0.
I20260808 02:16:48.862032 15781 raft_consensus.cc:504] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75639ad976e3488ea4bcb11509760102" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34471 } } peers { permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 } } peers { permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 } }
I20260808 02:16:48.862160 15781 leader_election.cc:288] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [CANDIDATE]: Term 1 election: Requested vote from peers 979e7f11ef774efd8c97338516687977 (127.19.58.195:42771), 872eaabce5c5475bb867d7bdb90b8b77 (127.19.58.194:46119)
I20260808 02:16:48.862305 15687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd29b59fddf4a66841105fada3d10cd" candidate_uuid: "75639ad976e3488ea4bcb11509760102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "979e7f11ef774efd8c97338516687977"
I20260808 02:16:48.862371 15485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd29b59fddf4a66841105fada3d10cd" candidate_uuid: "75639ad976e3488ea4bcb11509760102" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872eaabce5c5475bb867d7bdb90b8b77"
I20260808 02:16:48.862387 15687 raft_consensus.cc:3037] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:48.862453 15485 raft_consensus.cc:3037] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:48.863095 15687 raft_consensus.cc:2454] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75639ad976e3488ea4bcb11509760102 in term 1.
I20260808 02:16:48.863232 15485 raft_consensus.cc:2454] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75639ad976e3488ea4bcb11509760102 in term 1.
I20260808 02:16:48.863292 15198 leader_election.cc:302] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75639ad976e3488ea4bcb11509760102, 979e7f11ef774efd8c97338516687977; no voters:
I20260808 02:16:48.863389 15781 raft_consensus.cc:2781] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:48.863536 15781 raft_consensus.cc:686] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [term 1 LEADER]: Becoming Leader. State: Replica: 75639ad976e3488ea4bcb11509760102, State: Running, Role: LEADER
I20260808 02:16:48.863626 15781 consensus_queue.cc:237] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75639ad976e3488ea4bcb11509760102" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34471 } } peers { permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 } } peers { permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 } }
I20260808 02:16:48.864446 15085 catalog_manager.cc:5495] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75639ad976e3488ea4bcb11509760102 (127.19.58.193). New cstate: current_term: 1 leader_uuid: "75639ad976e3488ea4bcb11509760102" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75639ad976e3488ea4bcb11509760102" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 34471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 } health_report { overall_health: UNKNOWN } } }
W20260808 02:16:48.883795 15750 tablet.cc:2369] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 02:16:48.897794 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.895428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42436 (local address 127.19.58.193:34471)
0808 02:16:48.895505 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:48.895506 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:48.895515 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:16:48.895559 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:48.895559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:48.895578 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:48.895643 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:48.895808 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.896302 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.897045 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:48.897248 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:48.897303 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:48.897473 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:48.897581 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:48.897653 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:48.897692 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 02:16:48.897713 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:16:48.899626 15485 raft_consensus.cc:1261] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77 [term 1 FOLLOWER]: Refusing update from remote peer 75639ad976e3488ea4bcb11509760102: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:48.899624 15687 raft_consensus.cc:1261] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977 [term 1 FOLLOWER]: Refusing update from remote peer 75639ad976e3488ea4bcb11509760102: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:48.900024 15786 consensus_queue.cc:1035] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "872eaabce5c5475bb867d7bdb90b8b77" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:48.900147 15781 consensus_queue.cc:1035] T 3bd29b59fddf4a66841105fada3d10cd P 75639ad976e3488ea4bcb11509760102 [LEADER]: Connected to new peer: Peer: permanent_uuid: "979e7f11ef774efd8c97338516687977" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 42771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:48.902549 15791 mvcc.cc:205] Tried to move back new op lower bound from 7316092554850205696 to 7316092554706120704. Current Snapshot: MvccSnapshot[applied={T|T < 7316092554850205696}]
I20260808 02:16:48.903393 15793 mvcc.cc:205] Tried to move back new op lower bound from 7316092554850205696 to 7316092554706120704. Current Snapshot: MvccSnapshot[applied={T|T < 7316092554850205696}]
I20260808 02:16:48.905284 15086 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:48.905366 15086 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:48.906021 15086 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23684a56ca7c417d9f1265b9a04594ef: Sending DeleteTablet for 3 replicas of tablet 3bd29b59fddf4a66841105fada3d10cd
I20260808 02:16:48.906315 15265 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd29b59fddf4a66841105fada3d10cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:48 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:42420
I20260808 02:16:48.906363 15456 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd29b59fddf4a66841105fada3d10cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:48 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:56468
I20260808 02:16:48.906663 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 15169
I20260808 02:16:48.915959 15811 tablet_replica.cc:330] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: stopping tablet replica
I20260808 02:16:48.915930 15660 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd29b59fddf4a66841105fada3d10cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:48 UTC) from {username='kudu', principal='kudu/127.19.58.254@KRBTEST.COM'} at 127.0.0.1:50878
I20260808 02:16:48.916177 15811 raft_consensus.cc:2229] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:48.916329 15811 raft_consensus.cc:2258] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:48.916414 15814 tablet_replica.cc:330] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: stopping tablet replica
I20260808 02:16:48.916640 15814 raft_consensus.cc:2229] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:48.916822 15814 raft_consensus.cc:2258] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:48.917018 15811 ts_tablet_manager.cc:1905] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:48.917335 15814 ts_tablet_manager.cc:1905] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:48.918838 15811 ts_tablet_manager.cc:1918] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:48.918936 15811 log.cc:1196] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/3bd29b59fddf4a66841105fada3d10cd
I20260808 02:16:48.918959 15814 ts_tablet_manager.cc:1918] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:48.919015 15814 log.cc:1196] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/3bd29b59fddf4a66841105fada3d10cd
I20260808 02:16:48.919297 15811 ts_tablet_manager.cc:1939] T 3bd29b59fddf4a66841105fada3d10cd P 872eaabce5c5475bb867d7bdb90b8b77: Deleting consensus metadata
I20260808 02:16:48.919297 15814 ts_tablet_manager.cc:1939] T 3bd29b59fddf4a66841105fada3d10cd P 979e7f11ef774efd8c97338516687977: Deleting consensus metadata
I20260808 02:16:48.919785 15072 catalog_manager.cc:4841] TS 872eaabce5c5475bb867d7bdb90b8b77 (127.19.58.194:46119): tablet 3bd29b59fddf4a66841105fada3d10cd (table test-table [id=935e4b452d994f49bc9004239bdcf9c7]) successfully deleted
I20260808 02:16:48.919839 15070 catalog_manager.cc:4841] TS 979e7f11ef774efd8c97338516687977 (127.19.58.195:42771): tablet 3bd29b59fddf4a66841105fada3d10cd (table test-table [id=935e4b452d994f49bc9004239bdcf9c7]) successfully deleted
W20260808 02:16:48.920562 15071 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:34471 (error 108)
I20260808 02:16:48.921032 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:48.920912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:34471 (local address 127.0.0.1:42438)
0808 02:16:48.920964 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:48.920999 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 02:16:48.921118 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 15364
W20260808 02:16:48.921167 15071 catalog_manager.cc:4584] TS 75639ad976e3488ea4bcb11509760102 (127.19.58.193:34471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3bd29b59fddf4a66841105fada3d10cd: Network error: Client connection negotiation failed: client connection to 127.19.58.193:34471: connect: Connection refused (error 111)
I20260808 02:16:48.936702 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 15568
I20260808 02:16:48.944519 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 15049
2026-08-08T02:16:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2172 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T02:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:48Z Disabled control of system clock
I20260808 02:16:48.974817 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:44921
--webserver_interface=127.19.58.254
--webserver_port=0
--builtin_ntp_servers=127.19.58.212:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:44921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:49.055090 15819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:49.055291 15819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:49.055330 15819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:49.056452 15819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:49.056490 15819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:49.056502 15819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:49.056514 15819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:49.056524 15819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:49.057765 15819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:44921
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:44921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:49.057958 15819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:49.058187 15819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:49.062907 15830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:49.062889 15832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:49.062903 15827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:49.063494 15819 server_base.cc:1034] running on GCE node
I20260808 02:16:49.063784 15819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:49.064062 15819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:49.065227 15819 hybrid_clock.cc:648] HybridClock initialized: now 1786155409065193 us; error 50 us; skew 500 ppm
I20260808 02:16:49.066721 15819 webserver.cc:460] Webserver started at http://127.19.58.254:38593/ using document root <none> and password file <none>
I20260808 02:16:49.066957 15819 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:49.067004 15819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:49.067144 15819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:49.068271 15819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "730947732ee94c869a917588baca702c"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "093a3c9fa67cd49750395c26acbd2fcf"
server_key_iv: "4c88abe96fa5aee59c540920f581ceff"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.068621 15819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "730947732ee94c869a917588baca702c"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "093a3c9fa67cd49750395c26acbd2fcf"
server_key_iv: "4c88abe96fa5aee59c540920f581ceff"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.069802 15819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 02:16:49.071141 15843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.071314 15819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:49.071401 15819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "730947732ee94c869a917588baca702c"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "093a3c9fa67cd49750395c26acbd2fcf"
server_key_iv: "4c88abe96fa5aee59c540920f581ceff"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.071478 15819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:49.088510 15819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:49.088795 15819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:49.088907 15819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:49.092756 15819 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:44921
I20260808 02:16:49.092821 15912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:44921 every 8 connection(s)
I20260808 02:16:49.093130 15819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:49.094097 15913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:49.096447 15913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c: Bootstrap starting.
I20260808 02:16:49.096993 15913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:49.097267 15913 log.cc:824] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:49.097901 15913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c: No bootstrap required, opened a new log
I20260808 02:16:49.098994 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 15819
I20260808 02:16:49.099076 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:49.099243 19691 external_mini_cluster.cc:1428] Setting key 231016b58c56febd7a13760c869705e5
I20260808 02:16:49.099532 15913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730947732ee94c869a917588baca702c" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44921 } }
I20260808 02:16:49.099687 15913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:49.099746 15913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730947732ee94c869a917588baca702c, State: Initialized, Role: FOLLOWER
I20260808 02:16:49.099871 15913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730947732ee94c869a917588baca702c" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44921 } }
I20260808 02:16:49.099958 15913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:49.099995 15913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:49.100054 15913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:49.100721 15913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730947732ee94c869a917588baca702c" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44921 } }
I20260808 02:16:49.100836 15913 leader_election.cc:302] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 730947732ee94c869a917588baca702c; no voters:
I20260808 02:16:49.100996 15913 leader_election.cc:288] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:49.101030 15919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:49.101164 15919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [term 1 LEADER]: Becoming Leader. State: Replica: 730947732ee94c869a917588baca702c, State: Running, Role: LEADER
I20260808 02:16:49.101243 15913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:49.101264 15919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730947732ee94c869a917588baca702c" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44921 } }
I20260808 02:16:49.101593 15921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "730947732ee94c869a917588baca702c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730947732ee94c869a917588baca702c" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44921 } } }
I20260808 02:16:49.101680 15921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:49.101583 15919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 730947732ee94c869a917588baca702c. Latest consensus state: current_term: 1 leader_uuid: "730947732ee94c869a917588baca702c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "730947732ee94c869a917588baca702c" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 44921 } } }
I20260808 02:16:49.101864 15919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:49.101985 15929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:49.102646 15929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:49.104182 15929 catalog_manager.cc:1347] Generated new cluster ID: 2f234a9106ca4db1a4a4eda6e4c965ad
I20260808 02:16:49.104238 15929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:49.105711 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.099761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49374 (local address 127.19.58.254:44921)
0808 02:16:49.100003 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.100012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.100045 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:16:49.100157 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.100164 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.100234 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.100392 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:49.100591 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.102311 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.103384 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.104008 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.104122 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.105001 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:49.105006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:49.105006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:49.105218 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:49.105261 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:49.105275 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:49.105350 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.105415 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.105494 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":102,"threads_started":1}
I20260808 02:16:49.115742 15929 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:49.116245 15929 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:49.122699 15929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c: Generated new TSK 0
I20260808 02:16:49.122896 15929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:49.130293 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.193:0
--local_ip_for_outbound_sockets=127.19.58.193
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:44921
--builtin_ntp_servers=127.19.58.212:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:49.212065 15944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:49.212275 15944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:49.212297 15944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:49.213410 15944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:49.213447 15944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:49.213469 15944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.193
I20260808 02:16:49.214758 15944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.58.193
--webserver_port=0
--tserver_master_addrs=127.19.58.254:44921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.193
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:49.214962 15944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:49.215236 15944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:49.219993 15952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:49.220005 15956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:49.220121 15944 server_base.cc:1034] running on GCE node
W20260808 02:16:49.220237 15959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:49.220445 15944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:49.220692 15944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:49.221836 15944 hybrid_clock.cc:648] HybridClock initialized: now 1786155409221816 us; error 40 us; skew 500 ppm
I20260808 02:16:49.223228 15944 webserver.cc:460] Webserver started at http://127.19.58.193:42991/ using document root <none> and password file <none>
I20260808 02:16:49.223460 15944 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:49.223526 15944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:49.223642 15944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:49.224928 15944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/data/instance:
uuid: "f278ddd610524bf3918e5e1e33107327"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "faceed7f8090defd2dc3be3921058ad0"
server_key_iv: "68addbb22605d0fbb04f3fffb1fbdfc2"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.225307 15944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance:
uuid: "f278ddd610524bf3918e5e1e33107327"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "faceed7f8090defd2dc3be3921058ad0"
server_key_iv: "68addbb22605d0fbb04f3fffb1fbdfc2"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.226907 15944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:16:49.227540 15966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.227687 15944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:49.227787 15944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/wal
uuid: "f278ddd610524bf3918e5e1e33107327"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "faceed7f8090defd2dc3be3921058ad0"
server_key_iv: "68addbb22605d0fbb04f3fffb1fbdfc2"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.227846 15944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:49.239892 15944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:49.240211 15944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:49.240350 15944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:49.240593 15944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:49.240922 15944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:49.240964 15944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.240995 15944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:49.241012 15944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.246414 15944 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.193:42329
I20260808 02:16:49.246508 16128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.193:42329 every 8 connection(s)
I20260808 02:16:49.246766 15944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/data/info.pb
I20260808 02:16:49.251739 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.247426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.193:48173 (local address 127.19.58.254:44921)
0808 02:16:49.247491 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.247493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.247727 (+ 234us) server_negotiation.cc:407] Connection header received
0808 02:16:49.248288 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.248289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.248305 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.248358 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:49.249013 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.249523 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.250468 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.250671 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.250727 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.251454 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:49.251456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:49.251456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:49.251461 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:49.251478 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:49.251482 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:49.251526 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.251605 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.251637 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:16:49.251915 16132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.247408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44921 (local address 127.19.58.193:48173)
0808 02:16:49.247577 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.247620 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:49.248180 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:49.248426 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:49.248430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:49.248439 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:49.248913 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.248918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.249607 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.249608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.250373 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.250375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.250741 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.250742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.250866 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.251319 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:49.251319 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:49.251346 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:49.251366 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:49.251533 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:49.251539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:49.251542 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:49.251601 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:16:49.251728 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260808 02:16:49.252401 16129 heartbeater.cc:347] Connected to a master server at 127.19.58.254:44921
I20260808 02:16:49.252483 16129 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:49.252660 16129 heartbeater.cc:511] Master 127.19.58.254:44921 requested a full tablet report, sending...
I20260808 02:16:49.253125 15867 ts_manager.cc:194] Registered new tserver with Master: f278ddd610524bf3918e5e1e33107327 (127.19.58.193:42329)
I20260808 02:16:49.253746 15867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.193:48173
I20260808 02:16:49.255893 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 15944
I20260808 02:16:49.255987 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-0/wal/instance
I20260808 02:16:49.256166 19691 external_mini_cluster.cc:1428] Setting key d0e4c755aabaf4d707e994130b2fa0fa
I20260808 02:16:49.256963 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.194:0
--local_ip_for_outbound_sockets=127.19.58.194
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:44921
--builtin_ntp_servers=127.19.58.212:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:49.336076 16135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:49.336262 16135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:49.336282 16135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:49.337370 16135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:49.337409 16135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:49.337432 16135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.194
I20260808 02:16:49.338717 16135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.58.194
--webserver_port=0
--tserver_master_addrs=127.19.58.254:44921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.194
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:49.338917 16135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:49.339174 16135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:49.343858 16146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:49.343884 16142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:49.343884 16144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:49.343918 16135 server_base.cc:1034] running on GCE node
I20260808 02:16:49.344197 16135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:49.344398 16135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:49.345532 16135 hybrid_clock.cc:648] HybridClock initialized: now 1786155409345508 us; error 42 us; skew 500 ppm
I20260808 02:16:49.346606 16135 webserver.cc:460] Webserver started at http://127.19.58.194:40551/ using document root <none> and password file <none>
I20260808 02:16:49.346805 16135 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:49.346853 16135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:49.347002 16135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:49.348042 16135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/data/instance:
uuid: "c80b724400074c64aed7bdcefc11fde9"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "b13b8831903cce6e2ebbdef54fe0eb60"
server_key_iv: "fa64afd9a6e37a12557272fda3f7eed1"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.348366 16135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance:
uuid: "c80b724400074c64aed7bdcefc11fde9"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "b13b8831903cce6e2ebbdef54fe0eb60"
server_key_iv: "fa64afd9a6e37a12557272fda3f7eed1"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.349593 16135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:16:49.350245 16155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.350389 16135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:49.350468 16135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/wal
uuid: "c80b724400074c64aed7bdcefc11fde9"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "b13b8831903cce6e2ebbdef54fe0eb60"
server_key_iv: "fa64afd9a6e37a12557272fda3f7eed1"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.350528 16135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:49.359329 16135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:49.359596 16135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:49.359755 16135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:49.359980 16135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:49.360293 16135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:49.360327 16135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.360353 16135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:49.360371 16135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.365832 16135 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.194:43933
I20260808 02:16:49.365870 16323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.194:43933 every 8 connection(s)
I20260808 02:16:49.366184 16135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/data/info.pb
I20260808 02:16:49.371065 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.366881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.194:36391 (local address 127.19.58.254:44921)
0808 02:16:49.366940 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.366942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.367180 (+ 238us) server_negotiation.cc:407] Connection header received
0808 02:16:49.367684 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.367685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.367698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.367765 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:49.368417 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.368919 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.369839 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.370044 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.370099 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.370805 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:49.370807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:49.370807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:49.370812 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:49.370827 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:49.370831 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:49.370877 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.370953 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.370983 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 02:16:49.371250 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.366862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44921 (local address 127.19.58.194:36391)
0808 02:16:49.367054 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.367097 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:49.367579 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:49.367833 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:49.367837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:49.367847 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:49.368326 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.368330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.369003 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.369004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.369753 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.369756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.370113 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.370114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.370235 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.370677 (+ 442us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:49.370678 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:49.370705 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:49.370725 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:49.370883 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:49.370888 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:49.370891 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:49.370976 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:16:49.371073 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260808 02:16:49.371696 16324 heartbeater.cc:347] Connected to a master server at 127.19.58.254:44921
I20260808 02:16:49.371815 16324 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:49.371961 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 16135
I20260808 02:16:49.371985 16324 heartbeater.cc:511] Master 127.19.58.254:44921 requested a full tablet report, sending...
I20260808 02:16:49.372045 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/wal/instance
I20260808 02:16:49.372257 19691 external_mini_cluster.cc:1428] Setting key 9b11a21bba16e4440491f4df65cac14a
I20260808 02:16:49.372349 15867 ts_manager.cc:194] Registered new tserver with Master: c80b724400074c64aed7bdcefc11fde9 (127.19.58.194:43933)
I20260808 02:16:49.372733 15867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.194:36391
I20260808 02:16:49.373203 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.58.195:0
--local_ip_for_outbound_sockets=127.19.58.195
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:44921
--builtin_ntp_servers=127.19.58.212:42555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 02:16:49.453086 16330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:49.453300 16330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:49.453333 16330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:49.454483 16330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:49.454522 16330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:49.454555 16330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.58.195
I20260808 02:16:49.455919 16330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:42555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.58.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.58.195
--webserver_port=0
--tserver_master_addrs=127.19.58.254:44921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.58.195
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:49.456141 16330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:49.456418 16330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:49.461256 16342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:49.461246 16340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:49.461256 16346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:49.461352 16330 server_base.cc:1034] running on GCE node
I20260808 02:16:49.461675 16330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:49.461897 16330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:49.463038 16330 hybrid_clock.cc:648] HybridClock initialized: now 1786155409463023 us; error 30 us; skew 500 ppm
I20260808 02:16:49.464203 16330 webserver.cc:460] Webserver started at http://127.19.58.195:46777/ using document root <none> and password file <none>
I20260808 02:16:49.464423 16330 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:49.464473 16330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:49.464601 16330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:49.465595 16330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/data/instance:
uuid: "23f79b92409e4c08854df62e4b3c2589"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "99a2eee71aba355b2ae8cb5cb364c69e"
server_key_iv: "9849f87133572777518a2cf0088f6d1d"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.465926 16330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance:
uuid: "23f79b92409e4c08854df62e4b3c2589"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "99a2eee71aba355b2ae8cb5cb364c69e"
server_key_iv: "9849f87133572777518a2cf0088f6d1d"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.467101 16330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 02:16:49.467749 16355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.467902 16330 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.467973 16330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/wal
uuid: "23f79b92409e4c08854df62e4b3c2589"
format_stamp: "Formatted at 2026-08-08 02:16:49 on dist-test-slave-1l3l"
server_key: "99a2eee71aba355b2ae8cb5cb364c69e"
server_key_iv: "9849f87133572777518a2cf0088f6d1d"
server_key_version: "encryptionkey@0"
I20260808 02:16:49.468037 16330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:49.481819 16330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:49.482134 16330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:49.482263 16330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:49.482482 16330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 02:16:49.482815 16330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 02:16:49.482856 16330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.482892 16330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 02:16:49.482929 16330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:49.488471 16330 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.195:37639
I20260808 02:16:49.488538 16546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.195:37639 every 8 connection(s)
I20260808 02:16:49.488827 16330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/data/info.pb
I20260808 02:16:49.493448 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.489508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:57081 (local address 127.19.58.254:44921)
0808 02:16:49.489568 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.489570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.489770 (+ 200us) server_negotiation.cc:407] Connection header received
0808 02:16:49.490230 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.490231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.490240 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.490295 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:49.490905 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.491393 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.492318 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.492507 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.492553 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.493239 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:49.493240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:49.493240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:49.493244 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:49.493259 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:49.493262 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:49.493300 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.493357 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.493381 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 02:16:49.493614 16552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.489475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:44921 (local address 127.19.58.195:57081)
0808 02:16:49.489657 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.489696 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:49.490156 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:49.490348 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:49.490351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:49.490359 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:16:49.490827 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.490831 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.491457 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.491458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.492261 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.492264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.492564 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.492565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.492666 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.493122 (+ 456us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 02:16:49.493122 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:16:49.493149 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 02:16:49.493170 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:16:49.493305 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:16:49.493307 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:16:49.493309 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 02:16:49.493353 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:16:49.493428 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260808 02:16:49.494041 16547 heartbeater.cc:347] Connected to a master server at 127.19.58.254:44921
I20260808 02:16:49.494127 16547 heartbeater.cc:464] Registering TS with master...
I20260808 02:16:49.494268 16547 heartbeater.cc:511] Master 127.19.58.254:44921 requested a full tablet report, sending...
I20260808 02:16:49.494599 15867 ts_manager.cc:194] Registered new tserver with Master: 23f79b92409e4c08854df62e4b3c2589 (127.19.58.195:37639)
I20260808 02:16:49.494937 15867 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.58.195:57081
I20260808 02:16:49.498292 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 16330
I20260808 02:16:49.498383 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/wal/instance
I20260808 02:16:49.498564 19691 external_mini_cluster.cc:1428] Setting key b388c4cd30901f7100c2e176994eecb4
I20260808 02:16:49.499369 19691 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 02:16:49.503470 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.500704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49376 (local address 127.19.58.254:44921)
0808 02:16:49.500758 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.500759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.500876 (+ 117us) server_negotiation.cc:407] Connection header received
0808 02:16:49.500934 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.500935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.500948 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.500993 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:49.501183 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.501683 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.502417 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.502612 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.502664 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.503262 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:49.503263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:49.503264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:49.503267 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:49.503280 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:49.503284 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:49.503321 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.503354 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.503374 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 02:16:49.504873 15867 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 02:16:49.506309 15867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 02:16:49.512187 16559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.508207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:43933 (local address 127.0.0.1:53042)
0808 02:16:49.508712 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.508720 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:49.508773 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:49.508933 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:49.508934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:49.508936 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:49.508984 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.508985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.509940 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.509941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.511061 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.511064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.511888 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.511889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.511971 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.511984 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 02:16:49.512058 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 02:16:49.512111 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260808 02:16:49.512198 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.508298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53042 (local address 127.19.58.194:43933)
0808 02:16:49.508666 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.508670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.508772 (+ 102us) server_negotiation.cc:407] Connection header received
0808 02:16:49.508835 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.508837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.508851 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.508931 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:49.509087 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.509850 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.511130 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.511807 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.511877 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.511970 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.512073 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.512115 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":123,"threads_started":1}
I20260808 02:16:49.512351 16562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.508295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38540 (local address 127.19.58.195:37639)
0808 02:16:49.508577 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.508581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.508591 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:16:49.508719 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.508722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.508736 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.508818 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:49.509126 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.510227 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.511273 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.511874 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.511941 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.512027 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.512229 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.512272 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":90,"threads_started":1}
I20260808 02:16:49.512503 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.508207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.195:37639 (local address 127.0.0.1:38540)
0808 02:16:49.508496 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.508536 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:49.508607 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:49.508856 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:49.508859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:49.508861 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:49.509045 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.509048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.510338 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.510339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.511196 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.511199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.512060 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.512061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.512114 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.512117 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:49.512161 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 02:16:49.512190 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":25,"thread_start_us":67,"threads_started":1}
I20260808 02:16:49.512946 16564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.508406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:42329 (local address 127.0.0.1:55376)
0808 02:16:49.508719 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.508729 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:49.508773 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:49.509024 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:49.509025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:49.509027 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:49.509062 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.509063 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.510314 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.510315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.511387 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.511390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.512728 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.512729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.512792 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.512797 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 02:16:49.512830 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 02:16:49.512867 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":26,"thread_start_us":78,"threads_started":1}
I20260808 02:16:49.512933 16234 tablet_service.cc:1467] Processing CreateTablet for tablet a2328e3027064810b635c77fab6919c0 (DEFAULT_TABLE table=test-table [id=04234ab0e1174156be6b6ff525cb7fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:49.513298 16234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2328e3027064810b635c77fab6919c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:49.513185 16440 tablet_service.cc:1467] Processing CreateTablet for tablet a2328e3027064810b635c77fab6919c0 (DEFAULT_TABLE table=test-table [id=04234ab0e1174156be6b6ff525cb7fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:49.513342 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.508537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.19.58.193:42329)
0808 02:16:49.508716 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.508719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.508824 (+ 105us) server_negotiation.cc:407] Connection header received
0808 02:16:49.508867 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.508869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.508884 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.508953 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:49.509133 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.510231 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.511469 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.512185 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.513098 (+ 913us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.513200 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.513225 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.513267 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260808 02:16:49.513540 16440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2328e3027064810b635c77fab6919c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:49.513940 16038 tablet_service.cc:1467] Processing CreateTablet for tablet a2328e3027064810b635c77fab6919c0 (DEFAULT_TABLE table=test-table [id=04234ab0e1174156be6b6ff525cb7fee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 02:16:49.514259 16038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2328e3027064810b635c77fab6919c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:49.516213 16566 tablet_bootstrap.cc:492] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: Bootstrap starting.
I20260808 02:16:49.516723 16567 tablet_bootstrap.cc:492] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: Bootstrap starting.
I20260808 02:16:49.516974 16566 tablet_bootstrap.cc:654] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:49.517330 16567 tablet_bootstrap.cc:654] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:49.517336 16568 tablet_bootstrap.cc:492] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327: Bootstrap starting.
I20260808 02:16:49.517294 16566 log.cc:824] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:49.517659 16567 log.cc:824] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:49.517944 16568 tablet_bootstrap.cc:654] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:49.518230 16566 tablet_bootstrap.cc:492] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: No bootstrap required, opened a new log
I20260808 02:16:49.518249 16568 log.cc:824] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:49.518285 16566 ts_tablet_manager.cc:1397] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:49.518488 16567 tablet_bootstrap.cc:492] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: No bootstrap required, opened a new log
I20260808 02:16:49.518553 16567 ts_tablet_manager.cc:1397] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:49.519135 16568 tablet_bootstrap.cc:492] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327: No bootstrap required, opened a new log
I20260808 02:16:49.519209 16568 ts_tablet_manager.cc:1397] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 02:16:49.520236 16566 raft_consensus.cc:348] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 } } peers { permanent_uuid: "23f79b92409e4c08854df62e4b3c2589" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37639 } } peers { permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 } }
I20260808 02:16:49.520363 16566 raft_consensus.cc:374] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:49.520414 16566 raft_consensus.cc:729] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f79b92409e4c08854df62e4b3c2589, State: Initialized, Role: FOLLOWER
I20260808 02:16:49.520517 16566 consensus_queue.cc:260] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 } } peers { permanent_uuid: "23f79b92409e4c08854df62e4b3c2589" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37639 } } peers { permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 } }
I20260808 02:16:49.520764 16566 ts_tablet_manager.cc:1428] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 02:16:49.520787 16547 heartbeater.cc:503] Master 127.19.58.254:44921 was elected leader, sending a full tablet report...
I20260808 02:16:49.520967 16567 raft_consensus.cc:348] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 } } peers { permanent_uuid: "23f79b92409e4c08854df62e4b3c2589" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37639 } } peers { permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 } }
I20260808 02:16:49.521121 16567 raft_consensus.cc:374] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:49.521159 16567 raft_consensus.cc:729] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80b724400074c64aed7bdcefc11fde9, State: Initialized, Role: FOLLOWER
I20260808 02:16:49.521270 16567 consensus_queue.cc:260] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 } } peers { permanent_uuid: "23f79b92409e4c08854df62e4b3c2589" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37639 } } peers { permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 } }
I20260808 02:16:49.521520 16567 ts_tablet_manager.cc:1428] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:49.521672 16324 heartbeater.cc:503] Master 127.19.58.254:44921 was elected leader, sending a full tablet report...
I20260808 02:16:49.521589 16568 raft_consensus.cc:348] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 } } peers { permanent_uuid: "23f79b92409e4c08854df62e4b3c2589" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37639 } } peers { permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 } }
I20260808 02:16:49.521725 16568 raft_consensus.cc:374] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:49.521767 16568 raft_consensus.cc:729] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f278ddd610524bf3918e5e1e33107327, State: Initialized, Role: FOLLOWER
I20260808 02:16:49.521870 16568 consensus_queue.cc:260] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 } } peers { permanent_uuid: "23f79b92409e4c08854df62e4b3c2589" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37639 } } peers { permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 } }
I20260808 02:16:49.522130 16568 ts_tablet_manager.cc:1428] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 02:16:49.522167 16129 heartbeater.cc:503] Master 127.19.58.254:44921 was elected leader, sending a full tablet report...
I20260808 02:16:49.534080 16574 raft_consensus.cc:482] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 02:16:49.534148 16574 raft_consensus.cc:504] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 } } peers { permanent_uuid: "23f79b92409e4c08854df62e4b3c2589" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37639 } } peers { permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 } }
I20260808 02:16:49.534422 16574 leader_election.cc:288] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c80b724400074c64aed7bdcefc11fde9 (127.19.58.194:43933), f278ddd610524bf3918e5e1e33107327 (127.19.58.193:42329)
I20260808 02:16:49.537092 16552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.534556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.194:43933 (local address 127.19.58.195:34457)
0808 02:16:49.534606 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.534617 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:49.534664 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:49.534790 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:49.534791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:49.534792 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:49.534856 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.534858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.535533 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.535534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.536349 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.536352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.536878 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.536879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.536944 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.536947 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 02:16:49.536997 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 02:16:49.537033 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 02:16:49.537106 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.534635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:34457 (local address 127.19.58.194:43933)
0808 02:16:49.534676 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.534678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.534685 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:16:49.534710 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.534710 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.534719 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.534785 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:49.534922 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.535447 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.536450 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.536808 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.536860 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.536867 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.537009 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.537045 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 02:16:49.537189 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.534645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.58.195:58455 (local address 127.19.58.193:42329)
0808 02:16:49.534682 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.534683 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.534783 (+ 100us) server_negotiation.cc:407] Connection header received
0808 02:16:49.534832 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.534833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.534841 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.534895 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 02:16:49.535060 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.535601 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.536516 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.536877 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.536945 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.536952 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.537099 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.537134 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 02:16:49.537199 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.534566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:42329 (local address 127.19.58.195:58455)
0808 02:16:49.534726 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.534735 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:16:49.534785 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:16:49.534910 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:16:49.534911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:16:49.534912 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 02:16:49.534974 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.534976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.535673 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.535673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.536438 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:16:49.536440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.536953 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.536953 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:16:49.537028 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.537032 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 02:16:49.537087 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 02:16:49.537120 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":12,"thread_start_us":84,"threads_started":1}
I20260808 02:16:49.537410 16257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2328e3027064810b635c77fab6919c0" candidate_uuid: "23f79b92409e4c08854df62e4b3c2589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c80b724400074c64aed7bdcefc11fde9" is_pre_election: true
I20260808 02:16:49.537459 16060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2328e3027064810b635c77fab6919c0" candidate_uuid: "23f79b92409e4c08854df62e4b3c2589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f278ddd610524bf3918e5e1e33107327" is_pre_election: true
I20260808 02:16:49.537535 16257 raft_consensus.cc:2454] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23f79b92409e4c08854df62e4b3c2589 in term 0.
I20260808 02:16:49.537567 16060 raft_consensus.cc:2454] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23f79b92409e4c08854df62e4b3c2589 in term 0.
I20260808 02:16:49.537748 16358 leader_election.cc:302] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f79b92409e4c08854df62e4b3c2589, f278ddd610524bf3918e5e1e33107327; no voters:
I20260808 02:16:49.537905 16574 raft_consensus.cc:2781] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 02:16:49.537966 16574 raft_consensus.cc:482] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 02:16:49.537988 16574 raft_consensus.cc:3037] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:49.538647 16574 raft_consensus.cc:504] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 } } peers { permanent_uuid: "23f79b92409e4c08854df62e4b3c2589" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37639 } } peers { permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 } }
I20260808 02:16:49.538755 16574 leader_election.cc:288] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [CANDIDATE]: Term 1 election: Requested vote from peers c80b724400074c64aed7bdcefc11fde9 (127.19.58.194:43933), f278ddd610524bf3918e5e1e33107327 (127.19.58.193:42329)
I20260808 02:16:49.538901 16257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2328e3027064810b635c77fab6919c0" candidate_uuid: "23f79b92409e4c08854df62e4b3c2589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c80b724400074c64aed7bdcefc11fde9"
I20260808 02:16:49.538940 16060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2328e3027064810b635c77fab6919c0" candidate_uuid: "23f79b92409e4c08854df62e4b3c2589" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f278ddd610524bf3918e5e1e33107327"
I20260808 02:16:49.538976 16257 raft_consensus.cc:3037] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:49.538995 16060 raft_consensus.cc:3037] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:49.539633 16257 raft_consensus.cc:2454] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23f79b92409e4c08854df62e4b3c2589 in term 1.
I20260808 02:16:49.539638 16060 raft_consensus.cc:2454] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23f79b92409e4c08854df62e4b3c2589 in term 1.
I20260808 02:16:49.539817 16362 leader_election.cc:302] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f79b92409e4c08854df62e4b3c2589, c80b724400074c64aed7bdcefc11fde9; no voters:
I20260808 02:16:49.539915 16574 raft_consensus.cc:2781] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:49.540061 16574 raft_consensus.cc:686] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 1 LEADER]: Becoming Leader. State: Replica: 23f79b92409e4c08854df62e4b3c2589, State: Running, Role: LEADER
I20260808 02:16:49.540138 16574 consensus_queue.cc:237] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 } } peers { permanent_uuid: "23f79b92409e4c08854df62e4b3c2589" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37639 } } peers { permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 } }
I20260808 02:16:49.540822 15867 catalog_manager.cc:5495] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23f79b92409e4c08854df62e4b3c2589 (127.19.58.195). New cstate: current_term: 1 leader_uuid: "23f79b92409e4c08854df62e4b3c2589" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23f79b92409e4c08854df62e4b3c2589" member_type: VOTER last_known_addr { host: "127.19.58.195" port: 37639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 } health_report { overall_health: UNKNOWN } } }
I20260808 02:16:49.549293 16562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.547366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.19.58.195:37639)
0808 02:16:49.547430 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:49.547432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:49.547438 (+ 6us) server_negotiation.cc:407] Connection header received
0808 02:16:49.547462 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:49.547463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:49.547475 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:49.547528 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:16:49.547601 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.548053 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.548747 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:49.548919 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:49.548963 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:49.549064 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:16:49.549143 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:16:49.549192 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:49.549207 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 02:16:49.549220 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 02:16:49.550801 16060 raft_consensus.cc:1261] T a2328e3027064810b635c77fab6919c0 P f278ddd610524bf3918e5e1e33107327 [term 1 FOLLOWER]: Refusing update from remote peer 23f79b92409e4c08854df62e4b3c2589: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:49.550817 16257 raft_consensus.cc:1261] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9 [term 1 FOLLOWER]: Refusing update from remote peer 23f79b92409e4c08854df62e4b3c2589: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:16:49.551092 16574 consensus_queue.cc:1035] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c80b724400074c64aed7bdcefc11fde9" member_type: VOTER last_known_addr { host: "127.19.58.194" port: 43933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:49.551224 16581 consensus_queue.cc:1035] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f278ddd610524bf3918e5e1e33107327" member_type: VOTER last_known_addr { host: "127.19.58.193" port: 42329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:16:49.553996 16587 mvcc.cc:205] Tried to move back new op lower bound from 7316092557517860864 to 7316092557477031936. Current Snapshot: MvccSnapshot[applied={T|T < 7316092557517860864}]
I20260808 02:16:49.555286 16588 mvcc.cc:205] Tried to move back new op lower bound from 7316092557517860864 to 7316092557477031936. Current Snapshot: MvccSnapshot[applied={T|T < 7316092557517860864}]
I20260808 02:16:49.555599 15866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:49.555728 15866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 02:16:49.556464 15866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 730947732ee94c869a917588baca702c: Sending DeleteTablet for 3 replicas of tablet a2328e3027064810b635c77fab6919c0
I20260808 02:16:49.556757 16234 tablet_service.cc:1514] Processing DeleteTablet for tablet a2328e3027064810b635c77fab6919c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:49 UTC) from {username='slave'} at 127.0.0.1:53042
I20260808 02:16:49.556802 16038 tablet_service.cc:1514] Processing DeleteTablet for tablet a2328e3027064810b635c77fab6919c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:49 UTC) from {username='slave'} at 127.0.0.1:55376
I20260808 02:16:49.556973 16440 tablet_service.cc:1514] Processing DeleteTablet for tablet a2328e3027064810b635c77fab6919c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 02:16:49 UTC) from {username='slave'} at 127.0.0.1:38540
I20260808 02:16:49.557188 16605 tablet_replica.cc:330] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: stopping tablet replica
I20260808 02:16:49.557284 16605 raft_consensus.cc:2229] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 1 LEADER]: Raft consensus shutting down.
I20260808 02:16:49.557375 16605 raft_consensus.cc:2258] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:49.557472 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 15944
I20260808 02:16:49.560775 16605 ts_tablet_manager.cc:1905] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:49.562327 16605 ts_tablet_manager.cc:1918] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:49.562402 16605 log.cc:1196] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-2/wal/wals/a2328e3027064810b635c77fab6919c0
I20260808 02:16:49.562652 16605 ts_tablet_manager.cc:1939] T a2328e3027064810b635c77fab6919c0 P 23f79b92409e4c08854df62e4b3c2589: Deleting consensus metadata
I20260808 02:16:49.563053 15844 catalog_manager.cc:4841] TS 23f79b92409e4c08854df62e4b3c2589 (127.19.58.195:37639): tablet a2328e3027064810b635c77fab6919c0 (table test-table [id=04234ab0e1174156be6b6ff525cb7fee]) successfully deleted
I20260808 02:16:49.565377 16601 tablet_replica.cc:330] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: stopping tablet replica
I20260808 02:16:49.565492 16601 raft_consensus.cc:2229] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 02:16:49.565598 16601 raft_consensus.cc:2258] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 02:16:49.566071 16601 ts_tablet_manager.cc:1905] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 02:16:49.567649 16601 ts_tablet_manager.cc:1918] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 02:16:49.567751 16601 log.cc:1196] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: Deleting WAL directory at /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786155170542577-19691-0/minicluster-data/ts-1/wal/wals/a2328e3027064810b635c77fab6919c0
I20260808 02:16:49.567997 16601 ts_tablet_manager.cc:1939] T a2328e3027064810b635c77fab6919c0 P c80b724400074c64aed7bdcefc11fde9: Deleting consensus metadata
I20260808 02:16:49.568462 15846 catalog_manager.cc:4841] TS c80b724400074c64aed7bdcefc11fde9 (127.19.58.194:43933): tablet a2328e3027064810b635c77fab6919c0 (table test-table [id=04234ab0e1174156be6b6ff525cb7fee]) successfully deleted
W20260808 02:16:49.569469 15844 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.58.193:42329 (error 108)
I20260808 02:16:49.569779 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 16135
I20260808 02:16:49.571219 16564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:49.569890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.193:42329 (local address 127.0.0.1:55386)
0808 02:16:49.571139 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0808 02:16:49.571187 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.193:42329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1225}
W20260808 02:16:49.572217 15844 catalog_manager.cc:4584] TS f278ddd610524bf3918e5e1e33107327 (127.19.58.193:42329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a2328e3027064810b635c77fab6919c0: Network error: Client connection negotiation failed: client connection to 127.19.58.193:42329: connect: Connection refused (error 111)
I20260808 02:16:49.576794 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 16330
I20260808 02:16:49.585633 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 15819
2026-08-08T02:16:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (639 ms)
[----------] 6 tests from EncryptionPolicyTest (8716 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:49 dist-test-slave-1l3l krb5kdc[16611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:49 dist-test-slave-1l3l krb5kdc[16611](info): set up 2 sockets
Aug 08 02:16:49 dist-test-slave-1l3l krb5kdc[16611](info): commencing operation
krb5kdc: starting...
W20260808 02:16:50.698307 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:16:50 dist-test-slave-1l3l krb5kdc[16611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:16:50.816180 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:50.896041 16631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:50.896231 16631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:50.896252 16631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:50.897353 16631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:50.897401 16631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:50.897414 16631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:50.897426 16631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:50.897449 16631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:50.898880 16631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:50.899097 16631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:50.899315 16631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:50.904105 16636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:50.904148 16631 server_base.cc:1034] running on GCE node
W20260808 02:16:50.904120 16637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:50.904120 16639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:50.904542 16631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:50.904824 16631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:50.905980 16631 hybrid_clock.cc:648] HybridClock initialized: now 1786155410905955 us; error 38 us; skew 500 ppm
Aug 08 02:16:50 dist-test-slave-1l3l krb5kdc[16611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155410, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:50.908530 16631 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:16:50.908943 16631 webserver.cc:460] Webserver started at http://127.0.0.1:34093/ using document root <none> and password file <none>
I20260808 02:16:50.909148 16631 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:50.909204 16631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:50.909338 16631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:50.910348 16631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "0a6f02317e664e93bca8138870e28330"
format_stamp: "Formatted at 2026-08-08 02:16:50 on dist-test-slave-1l3l"
server_key: "a83ea30d7b3a137ac669b190663bb351"
server_key_iv: "bfc1d6239d958a5dffa701e179d53fed"
server_key_version: "encryptionkey@0"
I20260808 02:16:50.910683 16631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "0a6f02317e664e93bca8138870e28330"
format_stamp: "Formatted at 2026-08-08 02:16:50 on dist-test-slave-1l3l"
server_key: "a83ea30d7b3a137ac669b190663bb351"
server_key_iv: "bfc1d6239d958a5dffa701e179d53fed"
server_key_version: "encryptionkey@0"
I20260808 02:16:50.911969 16631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:16:50.912693 16647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:50.912901 16631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:50.912976 16631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "0a6f02317e664e93bca8138870e28330"
format_stamp: "Formatted at 2026-08-08 02:16:50 on dist-test-slave-1l3l"
server_key: "a83ea30d7b3a137ac669b190663bb351"
server_key_iv: "bfc1d6239d958a5dffa701e179d53fed"
server_key_version: "encryptionkey@0"
I20260808 02:16:50.913038 16631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:50.934381 16631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:50.936478 16631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:50.936623 16631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:50.942240 16631 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38215
I20260808 02:16:50.942281 16720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38215 every 8 connection(s)
I20260808 02:16:50.942607 16631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:50.943455 16722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:50.945708 16722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330: Bootstrap starting.
I20260808 02:16:50.946266 16722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:50.946527 16722 log.cc:824] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:50.947182 16722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330: No bootstrap required, opened a new log
I20260808 02:16:50.948766 16722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6f02317e664e93bca8138870e28330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260808 02:16:50.948916 16722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:50.948939 16722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a6f02317e664e93bca8138870e28330, State: Initialized, Role: FOLLOWER
I20260808 02:16:50.949074 16722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6f02317e664e93bca8138870e28330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260808 02:16:50.949134 16722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:50.949154 16722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:50.949199 16722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:50.949689 16722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6f02317e664e93bca8138870e28330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260808 02:16:50.949774 16722 leader_election.cc:302] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a6f02317e664e93bca8138870e28330; no voters:
I20260808 02:16:50.949922 16722 leader_election.cc:288] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:50.949975 16734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:50.950135 16734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [term 1 LEADER]: Becoming Leader. State: Replica: 0a6f02317e664e93bca8138870e28330, State: Running, Role: LEADER
I20260808 02:16:50.950158 16722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:50.950227 16734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6f02317e664e93bca8138870e28330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260808 02:16:50.950578 16737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a6f02317e664e93bca8138870e28330. Latest consensus state: current_term: 1 leader_uuid: "0a6f02317e664e93bca8138870e28330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6f02317e664e93bca8138870e28330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } } }
I20260808 02:16:50.950663 16737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:50.950840 16735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a6f02317e664e93bca8138870e28330" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a6f02317e664e93bca8138870e28330" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } } }
I20260808 02:16:50.950906 16735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:50.951071 16743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:50.951133 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 16631
I20260808 02:16:50.951246 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:50.951478 19691 external_mini_cluster.cc:1428] Setting key 8214892751103950ec439bba4c11997b
I20260808 02:16:50.951572 16743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:50.957144 16743 catalog_manager.cc:1347] Generated new cluster ID: 1bcea5e6f1bc424381376adee6f5710c
I20260808 02:16:50.957221 16743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:16:50 dist-test-slave-1l3l krb5kdc[16611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:16:50.961385 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:50.952214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54073 (local address 127.0.0.1:38215)
0808 02:16:50.952380 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:50.952389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:50.952421 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:16:50.952648 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:50.952653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:50.952695 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:50.952810 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:50.953483 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:50.954369 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:50.955122 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:50.955570 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:50.955669 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:50.958212 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:50.958217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:50.958217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:50.958229 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:50.959860 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:50.960223 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:50.960226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:50.960227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:50.960323 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:50.960538 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:50.960539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:50.960539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:50.960774 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:50.960934 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:50.961054 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 02:16:50.961155 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260808 02:16:50.962384 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:16:50.969548 16743 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:50.970238 16743 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:50.975131 16743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a6f02317e664e93bca8138870e28330: Generated new TSK 0
I20260808 02:16:50.975293 16743 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:50.975666 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:50.963952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33637 (local address 127.0.0.1:38215)
0808 02:16:50.963987 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:50.963989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:50.969247 (+ 5258us) server_negotiation.cc:407] Connection header received
0808 02:16:50.969312 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:50.969313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:50.969323 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:50.969452 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:50.970722 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:50.971200 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:50.971863 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:50.972073 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:50.972129 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:50.973197 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:50.973199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:50.973199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:50.973206 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:50.974431 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:50.974748 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:50.974749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:50.974749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:50.974797 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:50.975083 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:50.975085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:50.975085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:50.975211 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:50.975340 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:50.975547 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 02:16:50.975599 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:16:50.977830 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 16631
W20260808 02:16:50.988102 16609 connection.cc:535] client connection to 127.0.0.1:38215 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38215): Connection reset by peer (error 104)
2026-08-08T02:16:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:51 dist-test-slave-1l3l krb5kdc[16775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:51 dist-test-slave-1l3l krb5kdc[16775](info): set up 2 sockets
Aug 08 02:16:51 dist-test-slave-1l3l krb5kdc[16775](info): commencing operation
krb5kdc: starting...
W20260808 02:16:52.095605 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:16:52 dist-test-slave-1l3l krb5kdc[16775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:16:52.210510 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37541
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:52.292996 16790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:52.293201 16790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:52.293223 16790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:52.294334 16790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:52.294432 16790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:52.294447 16790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:52.294459 16790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:52.294469 16790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:52.296017 16790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37541
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:52.296241 16790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:52.296454 16790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:52.301157 16803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:52.301277 16790 server_base.cc:1034] running on GCE node
W20260808 02:16:52.301143 16800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:52.301144 16801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:52.301651 16790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:52.301964 16790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:52.303123 16790 hybrid_clock.cc:648] HybridClock initialized: now 1786155412303110 us; error 29 us; skew 500 ppm
Aug 08 02:16:52 dist-test-slave-1l3l krb5kdc[16775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155412, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:52.305529 16790 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:16:52.305974 16790 webserver.cc:460] Webserver started at http://127.0.0.1:35767/ using document root <none> and password file <none>
I20260808 02:16:52.306180 16790 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:52.306228 16790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:52.306353 16790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:52.307364 16790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c"
format_stamp: "Formatted at 2026-08-08 02:16:52 on dist-test-slave-1l3l"
server_key: "8bd927e4a68dd9d9bab3f33cffea9cab"
server_key_iv: "0ba2a7aaf1b358f456c4de3b4458ee4e"
server_key_version: "encryptionkey@0"
I20260808 02:16:52.307740 16790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c"
format_stamp: "Formatted at 2026-08-08 02:16:52 on dist-test-slave-1l3l"
server_key: "8bd927e4a68dd9d9bab3f33cffea9cab"
server_key_iv: "0ba2a7aaf1b358f456c4de3b4458ee4e"
server_key_version: "encryptionkey@0"
I20260808 02:16:52.308984 16790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:16:52.309628 16816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:52.309770 16790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:16:52.309868 16790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c"
format_stamp: "Formatted at 2026-08-08 02:16:52 on dist-test-slave-1l3l"
server_key: "8bd927e4a68dd9d9bab3f33cffea9cab"
server_key_iv: "0ba2a7aaf1b358f456c4de3b4458ee4e"
server_key_version: "encryptionkey@0"
I20260808 02:16:52.309935 16790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:52.328558 16790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:52.330441 16790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:52.330588 16790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:52.334676 16790 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37541
I20260808 02:16:52.334717 16899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37541 every 8 connection(s)
I20260808 02:16:52.335049 16790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:52.335610 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 16790
I20260808 02:16:52.335759 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:52.335978 19691 external_mini_cluster.cc:1428] Setting key a1f30dce8ca7f3f39099d916d5c0b681
I20260808 02:16:52.336094 16900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:52.339111 16900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c: Bootstrap starting.
I20260808 02:16:52.339780 16900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:52.340097 16900 log.cc:824] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:52.340865 16900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c: No bootstrap required, opened a new log
I20260808 02:16:52.341512 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:52.336500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42347 (local address 127.0.0.1:37541)
0808 02:16:52.336694 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:52.336702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:52.336728 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:16:52.337156 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:52.337163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:52.337210 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:52.337319 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:52.337540 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:52.338409 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:52.339472 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:52.339986 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:52.340088 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:52.340856 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:52.340860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:52.340860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:52.341046 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:52.341086 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:52.341097 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:52.341161 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:52.341275 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 02:16:52.341300 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260808 02:16:52.342518 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:16:52.342999 16900 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37541 } }
I20260808 02:16:52.343130 16900 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:52.343151 16900 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f7e1f3196e4ee5acea92fe1e18e66c, State: Initialized, Role: FOLLOWER
I20260808 02:16:52.343261 16900 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37541 } }
I20260808 02:16:52.343326 16900 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:52.343348 16900 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:52.343392 16900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:52.344072 16900 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37541 } }
I20260808 02:16:52.344170 16900 leader_election.cc:302] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6f7e1f3196e4ee5acea92fe1e18e66c; no voters:
I20260808 02:16:52.344316 16900 leader_election.cc:288] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:52.344368 16916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:52.344527 16916 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [term 1 LEADER]: Becoming Leader. State: Replica: a6f7e1f3196e4ee5acea92fe1e18e66c, State: Running, Role: LEADER
I20260808 02:16:52.344543 16900 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:52.344620 16916 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37541 } }
I20260808 02:16:52.344964 16918 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37541 } } }
I20260808 02:16:52.344978 16919 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6f7e1f3196e4ee5acea92fe1e18e66c. Latest consensus state: current_term: 1 leader_uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f7e1f3196e4ee5acea92fe1e18e66c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37541 } } }
I20260808 02:16:52.345129 16918 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:52.345140 16919 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:52.345578 16929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:52.346168 16929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:52.348032 16929 catalog_manager.cc:1347] Generated new cluster ID: f5d17a355b1a41cabb9a191cbca16c2c
I20260808 02:16:52.348083 16929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:52.349458 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:52.343892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33937 (local address 127.0.0.1:37541)
0808 02:16:52.343936 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:52.343938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:52.345924 (+ 1986us) server_negotiation.cc:407] Connection header received
0808 02:16:52.345980 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:52.345981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:52.345991 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:52.346050 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:52.346182 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:52.346832 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:52.348138 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:52.348439 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:52.348482 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:52.349227 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:52.349229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:52.349230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:52.349234 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:52.349250 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:52.349254 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:52.349307 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:52.349377 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 02:16:52.349387 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 02:16:52.372275 16929 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:52.372846 16929 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:52.379666 16929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6f7e1f3196e4ee5acea92fe1e18e66c: Generated new TSK 0
I20260808 02:16:52.379889 16929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:52.384783 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 16790
W20260808 02:16:52.392539 16771 connection.cc:535] client connection to 127.0.0.1:37541 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37541): Connection reset by peer (error 104)
2026-08-08T02:16:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:52 dist-test-slave-1l3l krb5kdc[16944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:52 dist-test-slave-1l3l krb5kdc[16944](info): set up 2 sockets
Aug 08 02:16:52 dist-test-slave-1l3l krb5kdc[16944](info): commencing operation
krb5kdc: starting...
W20260808 02:16:53.509779 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:16:53 dist-test-slave-1l3l krb5kdc[16944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:16:53.625001 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:53.704489 16962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:53.704680 16962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:53.704710 16962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:53.705801 16962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:53.705842 16962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:53.705862 16962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:53.705878 16962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:53.705893 16962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:53.707309 16962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40875
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:53.707556 16962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:53.707790 16962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:53.712531 16976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:53.712543 16974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:53.712642 16962 server_base.cc:1034] running on GCE node
W20260808 02:16:53.712754 16971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:53.712965 16962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:53.713205 16962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:53.714339 16962 hybrid_clock.cc:648] HybridClock initialized: now 1786155413714327 us; error 34 us; skew 500 ppm
Aug 08 02:16:53 dist-test-slave-1l3l krb5kdc[16944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155413, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:53.720525 16962 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:16:53.720958 16962 webserver.cc:460] Webserver started at http://127.0.0.1:35035/ using document root <none> and password file <none>
I20260808 02:16:53.721135 16962 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:53.721176 16962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:53.721266 16962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:53.722236 16962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "35dcc87fd9d946ed8867737ebe74f711"
format_stamp: "Formatted at 2026-08-08 02:16:53 on dist-test-slave-1l3l"
server_key: "37569f5976b10719daf2116372fc8b0d"
server_key_iv: "3b54b8fa3ea9efce11ba3c483ca3797b"
server_key_version: "encryptionkey@0"
I20260808 02:16:53.722530 16962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "35dcc87fd9d946ed8867737ebe74f711"
format_stamp: "Formatted at 2026-08-08 02:16:53 on dist-test-slave-1l3l"
server_key: "37569f5976b10719daf2116372fc8b0d"
server_key_iv: "3b54b8fa3ea9efce11ba3c483ca3797b"
server_key_version: "encryptionkey@0"
I20260808 02:16:53.723688 16962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:16:53.724392 16985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:53.724563 16962 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:16:53.724644 16962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "35dcc87fd9d946ed8867737ebe74f711"
format_stamp: "Formatted at 2026-08-08 02:16:53 on dist-test-slave-1l3l"
server_key: "37569f5976b10719daf2116372fc8b0d"
server_key_iv: "3b54b8fa3ea9efce11ba3c483ca3797b"
server_key_version: "encryptionkey@0"
I20260808 02:16:53.724696 16962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:53.727748 16962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:53.729679 16962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:53.729781 16962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:53.733422 16962 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40875
I20260808 02:16:53.733479 17075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40875 every 8 connection(s)
I20260808 02:16:53.733736 16962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:53.734642 17076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:53.736888 17076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711: Bootstrap starting.
I20260808 02:16:53.737438 17076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:53.737694 17076 log.cc:824] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:53.738306 17076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711: No bootstrap required, opened a new log
I20260808 02:16:53.739445 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 16962
I20260808 02:16:53.739564 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:53.739761 19691 external_mini_cluster.cc:1428] Setting key 1d7cb5735c9b2d33f0d83b4958d6a127
I20260808 02:16:53.740182 17076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35dcc87fd9d946ed8867737ebe74f711" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40875 } }
I20260808 02:16:53.740353 17076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:53.740391 17076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dcc87fd9d946ed8867737ebe74f711, State: Initialized, Role: FOLLOWER
I20260808 02:16:53.740518 17076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35dcc87fd9d946ed8867737ebe74f711" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40875 } }
I20260808 02:16:53.740588 17076 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:53.740618 17076 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:53.740664 17076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:53.741299 17076 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35dcc87fd9d946ed8867737ebe74f711" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40875 } }
I20260808 02:16:53.741403 17076 leader_election.cc:302] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35dcc87fd9d946ed8867737ebe74f711; no voters:
I20260808 02:16:53.741621 17082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:53.741597 17076 leader_election.cc:288] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:53.741772 17082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [term 1 LEADER]: Becoming Leader. State: Replica: 35dcc87fd9d946ed8867737ebe74f711, State: Running, Role: LEADER
I20260808 02:16:53.741798 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:53.740252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57069 (local address 127.0.0.1:40875)
0808 02:16:53.740430 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:53.740439 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:53.740474 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:16:53.740577 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:53.740582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:53.740604 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:53.740737 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:53.741320 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:53.741323 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:53.741323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:53.741326 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:53.741352 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:53.741362 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:53.741433 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:53.741501 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 02:16:53.741540 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260808 02:16:53.741854 17082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35dcc87fd9d946ed8867737ebe74f711" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40875 } }
I20260808 02:16:53.741923 17076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:53.742162 17082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35dcc87fd9d946ed8867737ebe74f711. Latest consensus state: current_term: 1 leader_uuid: "35dcc87fd9d946ed8867737ebe74f711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35dcc87fd9d946ed8867737ebe74f711" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40875 } } }
I20260808 02:16:53.742288 17082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:53.742573 17082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35dcc87fd9d946ed8867737ebe74f711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35dcc87fd9d946ed8867737ebe74f711" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40875 } } }
I20260808 02:16:53.742642 17082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:53.742657 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:16:53.742942 17091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:53.743386 17091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:53.745105 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:53.744036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52525 (local address 127.0.0.1:40875)
0808 02:16:53.744066 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:53.744068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:53.744202 (+ 134us) server_negotiation.cc:407] Connection header received
0808 02:16:53.744258 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:53.744258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:53.744263 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:53.744307 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:53.744890 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:53.744891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:53.744891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:53.744895 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:53.744909 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:53.744913 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:53.744940 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:53.745039 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 02:16:53.745051 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 02:16:53.745132 17091 catalog_manager.cc:1347] Generated new cluster ID: bcc9cc003f584fd297023c1612162c6a
I20260808 02:16:53.745200 17091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:53.752274 17091 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:53.752432 17091 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:53.758538 17091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35dcc87fd9d946ed8867737ebe74f711: Generated new TSK 0
I20260808 02:16:53.758689 17091 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:53.761229 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 16962
2026-08-08T02:16:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:53 dist-test-slave-1l3l krb5kdc[17119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:53 dist-test-slave-1l3l krb5kdc[17119](info): set up 2 sockets
Aug 08 02:16:53 dist-test-slave-1l3l krb5kdc[17119](info): commencing operation
krb5kdc: starting...
W20260808 02:16:54.894382 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:16:54 dist-test-slave-1l3l krb5kdc[17119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:16:55.006165 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:55.086730 17140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:55.086927 17140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:55.086958 17140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:55.088088 17140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:55.088128 17140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:55.088146 17140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:55.088161 17140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:55.088177 17140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:55.089651 17140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:55.089879 17140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:55.090076 17140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:55.094753 17147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:55.094815 17151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:55.094868 17148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:55.094909 17140 server_base.cc:1034] running on GCE node
I20260808 02:16:55.095152 17140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:55.095449 17140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:55.096594 17140 hybrid_clock.cc:648] HybridClock initialized: now 1786155415096572 us; error 33 us; skew 500 ppm
Aug 08 02:16:55 dist-test-slave-1l3l krb5kdc[17119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155415, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:55.099640 17140 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:16:55.100171 17140 webserver.cc:460] Webserver started at http://127.0.0.1:35055/ using document root <none> and password file <none>
I20260808 02:16:55.100345 17140 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:55.100384 17140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:55.100481 17140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:55.101449 17140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "eb423be14e394025ad6e76bf6c33fce4"
format_stamp: "Formatted at 2026-08-08 02:16:55 on dist-test-slave-1l3l"
server_key: "3cf550f37384eaa0429ebb603a9b4383"
server_key_iv: "b62a585b2073e52b2a1199d4f2612b1d"
server_key_version: "encryptionkey@0"
I20260808 02:16:55.101734 17140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "eb423be14e394025ad6e76bf6c33fce4"
format_stamp: "Formatted at 2026-08-08 02:16:55 on dist-test-slave-1l3l"
server_key: "3cf550f37384eaa0429ebb603a9b4383"
server_key_iv: "b62a585b2073e52b2a1199d4f2612b1d"
server_key_version: "encryptionkey@0"
I20260808 02:16:55.102917 17140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:16:55.103546 17165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:55.103688 17140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:55.103786 17140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "eb423be14e394025ad6e76bf6c33fce4"
format_stamp: "Formatted at 2026-08-08 02:16:55 on dist-test-slave-1l3l"
server_key: "3cf550f37384eaa0429ebb603a9b4383"
server_key_iv: "b62a585b2073e52b2a1199d4f2612b1d"
server_key_version: "encryptionkey@0"
I20260808 02:16:55.103840 17140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:55.113760 17140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:55.115361 17140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:55.115473 17140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:55.119086 17140 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38899
I20260808 02:16:55.119132 17248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38899 every 8 connection(s)
I20260808 02:16:55.119426 17140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:55.120252 17249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:55.120632 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 17140
I20260808 02:16:55.120714 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:55.120870 19691 external_mini_cluster.cc:1428] Setting key 16df7ad959aec08a68b4914a10b169a9
I20260808 02:16:55.122975 17249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4: Bootstrap starting.
I20260808 02:16:55.123677 17249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:55.124007 17249 log.cc:824] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:55.124670 17249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4: No bootstrap required, opened a new log
I20260808 02:16:55.126837 17249 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb423be14e394025ad6e76bf6c33fce4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } }
I20260808 02:16:55.126984 17249 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:55.127020 17249 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb423be14e394025ad6e76bf6c33fce4, State: Initialized, Role: FOLLOWER
I20260808 02:16:55.127148 17249 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb423be14e394025ad6e76bf6c33fce4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } }
I20260808 02:16:55.127223 17249 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:55.127260 17249 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:55.127308 17249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 02:16:55 dist-test-slave-1l3l krb5kdc[17119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:16:55.127969 17249 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb423be14e394025ad6e76bf6c33fce4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } }
I20260808 02:16:55.128069 17249 leader_election.cc:302] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb423be14e394025ad6e76bf6c33fce4; no voters:
I20260808 02:16:55.128237 17249 leader_election.cc:288] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:55.128278 17258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:55.128434 17258 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [term 1 LEADER]: Becoming Leader. State: Replica: eb423be14e394025ad6e76bf6c33fce4, State: Running, Role: LEADER
I20260808 02:16:55.128535 17249 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:55.128578 17258 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb423be14e394025ad6e76bf6c33fce4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } }
I20260808 02:16:55.128926 17258 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb423be14e394025ad6e76bf6c33fce4. Latest consensus state: current_term: 1 leader_uuid: "eb423be14e394025ad6e76bf6c33fce4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb423be14e394025ad6e76bf6c33fce4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } } }
I20260808 02:16:55.129132 17258 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:55.129071 17260 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb423be14e394025ad6e76bf6c33fce4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb423be14e394025ad6e76bf6c33fce4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } } }
I20260808 02:16:55.129491 17260 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:55.129613 17271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:55.130074 17271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:55.131830 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:55.121360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39137 (local address 127.0.0.1:38899)
0808 02:16:55.121538 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:55.121548 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:55.121581 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:16:55.122090 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:55.122099 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:55.122156 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:55.122297 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:55.122870 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:55.124243 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:55.125014 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:55.125609 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:55.125719 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:55.128069 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:55.128074 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:55.128074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:55.128085 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:55.130089 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:55.130495 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:55.130497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:55.130498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:55.130595 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:55.130831 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:55.130832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:55.130832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:55.131124 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:55.131272 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:55.131501 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 02:16:55.131630 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260808 02:16:55.131955 17271 catalog_manager.cc:1347] Generated new cluster ID: 26f4550d9afe4e6da6e1e50a264f7248
I20260808 02:16:55.132004 17271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:55.132669 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:16:55.142627 17254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:55.135965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51955 (local address 127.0.0.1:38899)
0808 02:16:55.136003 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:55.136004 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:55.136183 (+ 179us) server_negotiation.cc:407] Connection header received
0808 02:16:55.136211 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:55.136212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:55.136222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:55.136285 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:55.136823 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:55.137596 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:55.138511 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:55.138845 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:55.138929 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:55.140138 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:55.140139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:55.140139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:55.140146 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:55.141593 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:55.141942 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:55.141943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:55.141943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:55.141996 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:55.142159 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:55.142160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:55.142160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:55.142272 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:55.142358 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:55.142466 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 02:16:55.142537 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 02:16:55.143118 17271 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:55.143647 17271 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:55.153777 17271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb423be14e394025ad6e76bf6c33fce4: Generated new TSK 0
I20260808 02:16:55.153959 17271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:55.161530 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 17140
2026-08-08T02:16:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:55 dist-test-slave-1l3l krb5kdc[17300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:55 dist-test-slave-1l3l krb5kdc[17300](info): set up 2 sockets
Aug 08 02:16:55 dist-test-slave-1l3l krb5kdc[17300](info): commencing operation
krb5kdc: starting...
W20260808 02:16:56.271414 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:16:56 dist-test-slave-1l3l krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:16:56.382726 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45469
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:56.460875 17321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:56.461064 17321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:56.461090 17321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:56.462189 17321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:56.462230 17321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:56.462244 17321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:56.462267 17321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:56.462291 17321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:56.463737 17321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:56.463951 17321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:56.464152 17321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:56.468600 17338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:56.468636 17333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:56.468633 17334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:56.468770 17321 server_base.cc:1034] running on GCE node
I20260808 02:16:56.468982 17321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:56.469233 17321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:56.470367 17321 hybrid_clock.cc:648] HybridClock initialized: now 1786155416470342 us; error 38 us; skew 500 ppm
Aug 08 02:16:56 dist-test-slave-1l3l krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155416, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:56.472849 17321 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:16:56.473272 17321 webserver.cc:460] Webserver started at http://127.0.0.1:41663/ using document root <none> and password file <none>
I20260808 02:16:56.473466 17321 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:56.473513 17321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:56.473637 17321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:56.474592 17321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "b92fc5ead4f349f991580e4d8bad4fce"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-1l3l"
server_key: "c141c77e0ca10831003fb8426eba4c84"
server_key_iv: "49d229e81d83dbd2c012b08ccdfaa9e1"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.474913 17321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "b92fc5ead4f349f991580e4d8bad4fce"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-1l3l"
server_key: "c141c77e0ca10831003fb8426eba4c84"
server_key_iv: "49d229e81d83dbd2c012b08ccdfaa9e1"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.476187 17321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:16:56.476876 17348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:56.477023 17321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:16:56.477103 17321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "b92fc5ead4f349f991580e4d8bad4fce"
format_stamp: "Formatted at 2026-08-08 02:16:56 on dist-test-slave-1l3l"
server_key: "c141c77e0ca10831003fb8426eba4c84"
server_key_iv: "49d229e81d83dbd2c012b08ccdfaa9e1"
server_key_version: "encryptionkey@0"
I20260808 02:16:56.477169 17321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:56.488065 17321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:56.490033 17321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:56.490163 17321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:56.493920 17321 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45469
I20260808 02:16:56.493954 17430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45469 every 8 connection(s)
I20260808 02:16:56.494267 17321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:56.495064 17433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:56.496819 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 17321
I20260808 02:16:56.496930 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:56.497098 19691 external_mini_cluster.cc:1428] Setting key eb6bed54268b221b2a159268449066ae
I20260808 02:16:56.497344 17433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce: Bootstrap starting.
I20260808 02:16:56.498080 17433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:56.498402 17433 log.cc:824] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:56.499166 17433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce: No bootstrap required, opened a new log
I20260808 02:16:56.501358 17433 raft_consensus.cc:348] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92fc5ead4f349f991580e4d8bad4fce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45469 } }
I20260808 02:16:56.501536 17433 raft_consensus.cc:374] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:56.501575 17433 raft_consensus.cc:729] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92fc5ead4f349f991580e4d8bad4fce, State: Initialized, Role: FOLLOWER
I20260808 02:16:56.501739 17433 consensus_queue.cc:260] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92fc5ead4f349f991580e4d8bad4fce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45469 } }
I20260808 02:16:56.501828 17433 raft_consensus.cc:388] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:56.501868 17433 raft_consensus.cc:482] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:56.501914 17433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:56.502607 17433 raft_consensus.cc:504] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92fc5ead4f349f991580e4d8bad4fce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45469 } }
I20260808 02:16:56.502712 17433 leader_election.cc:302] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b92fc5ead4f349f991580e4d8bad4fce; no voters:
I20260808 02:16:56.502928 17439 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:56.502872 17433 leader_election.cc:288] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:56.503090 17439 raft_consensus.cc:686] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [term 1 LEADER]: Becoming Leader. State: Replica: b92fc5ead4f349f991580e4d8bad4fce, State: Running, Role: LEADER
I20260808 02:16:56.503226 17439 consensus_queue.cc:237] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92fc5ead4f349f991580e4d8bad4fce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45469 } }
I20260808 02:16:56.503275 17433 sys_catalog.cc:564] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:56.503571 17440 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [sys.catalog]: SysCatalogTable state changed. Reason: New leader b92fc5ead4f349f991580e4d8bad4fce. Latest consensus state: current_term: 1 leader_uuid: "b92fc5ead4f349f991580e4d8bad4fce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92fc5ead4f349f991580e4d8bad4fce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45469 } } }
I20260808 02:16:56.503584 17441 sys_catalog.cc:455] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b92fc5ead4f349f991580e4d8bad4fce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b92fc5ead4f349f991580e4d8bad4fce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45469 } } }
I20260808 02:16:56.503751 17440 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:56.503762 17441 sys_catalog.cc:458] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:56.504149 17450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:56.502872 17436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.497548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59155 (local address 127.0.0.1:45469)
0808 02:16:56.497724 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:56.497734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:56.497769 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:16:56.498319 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:56.498329 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:56.498391 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:56.498535 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:56.498735 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.500094 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.500846 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.501453 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.501562 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.502210 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:56.502214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:56.502215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:56.502416 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:56.502460 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:56.502474 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:56.502541 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:56.502591 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 02:16:56.502654 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260808 02:16:56.504639 17450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:56.505527 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:16:56.506348 17450 catalog_manager.cc:1347] Generated new cluster ID: 1e7f56bd7ef14b2fad3241c002a2b39c
I20260808 02:16:56.506450 17450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:56.510038 17436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:56.506865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42621 (local address 127.0.0.1:45469)
0808 02:16:56.506893 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:56.506895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:56.507022 (+ 127us) server_negotiation.cc:407] Connection header received
0808 02:16:56.507085 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:56.507085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:56.507094 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:56.507136 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:56.507251 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.507948 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.508912 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:56.509190 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:56.509241 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:56.509840 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:56.509841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:56.509841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:56.509845 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:56.509857 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:56.509861 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:56.509902 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:56.509953 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 02:16:56.509970 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 02:16:56.513031 17450 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:56.513491 17450 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:56.521380 17450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b92fc5ead4f349f991580e4d8bad4fce: Generated new TSK 0
I20260808 02:16:56.521540 17450 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:56.527580 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 17321
2026-08-08T02:16:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:56 dist-test-slave-1l3l krb5kdc[17479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:56 dist-test-slave-1l3l krb5kdc[17479](info): set up 2 sockets
Aug 08 02:16:56 dist-test-slave-1l3l krb5kdc[17479](info): commencing operation
krb5kdc: starting...
W20260808 02:16:57.642494 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:16:57 dist-test-slave-1l3l krb5kdc[17479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:16:57.752103 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35479
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:57.832423 17498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:57.832619 17498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:57.832640 17498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:57.833783 17498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:57.833827 17498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:57.833838 17498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:57.833850 17498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:57.833860 17498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:57.835316 17498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:57.835552 17498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:57.835781 17498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:57.840406 17505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:57.840432 17507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:57.840413 17504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:57.840605 17498 server_base.cc:1034] running on GCE node
I20260808 02:16:57.840816 17498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:57.841050 17498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:57.842185 17498 hybrid_clock.cc:648] HybridClock initialized: now 1786155417842169 us; error 31 us; skew 500 ppm
Aug 08 02:16:57 dist-test-slave-1l3l krb5kdc[17479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155417, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:57.844633 17498 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:16:57.845043 17498 webserver.cc:460] Webserver started at http://127.0.0.1:42167/ using document root <none> and password file <none>
I20260808 02:16:57.845244 17498 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:57.845285 17498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:57.845376 17498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:57.846362 17498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "3120e47f03164c868ef852fdaa891fe1"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-1l3l"
server_key: "1d2982de37cd949441c8b167658d807f"
server_key_iv: "a8ce875d674fa419d9a0c31778b01cdf"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.846653 17498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "3120e47f03164c868ef852fdaa891fe1"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-1l3l"
server_key: "1d2982de37cd949441c8b167658d807f"
server_key_iv: "a8ce875d674fa419d9a0c31778b01cdf"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.847958 17498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:16:57.848577 17517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:57.848727 17498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:57.848807 17498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "3120e47f03164c868ef852fdaa891fe1"
format_stamp: "Formatted at 2026-08-08 02:16:57 on dist-test-slave-1l3l"
server_key: "1d2982de37cd949441c8b167658d807f"
server_key_iv: "a8ce875d674fa419d9a0c31778b01cdf"
server_key_version: "encryptionkey@0"
I20260808 02:16:57.848868 17498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:57.852028 17498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:57.854022 17498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:57.854151 17498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:57.857988 17498 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35479
I20260808 02:16:57.858021 17592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35479 every 8 connection(s)
I20260808 02:16:57.858305 17498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:57.859231 17593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:57.861441 17593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1: Bootstrap starting.
I20260808 02:16:57.861967 17593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:57.862224 17593 log.cc:824] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:57.862838 17593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1: No bootstrap required, opened a new log
I20260808 02:16:57.864444 17593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3120e47f03164c868ef852fdaa891fe1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35479 } }
I20260808 02:16:57.864573 17593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:57.864593 17593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3120e47f03164c868ef852fdaa891fe1, State: Initialized, Role: FOLLOWER
I20260808 02:16:57.864717 17593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3120e47f03164c868ef852fdaa891fe1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35479 } }
I20260808 02:16:57.864773 17593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:57.864804 17593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:57.864848 17593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:57.865345 17593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3120e47f03164c868ef852fdaa891fe1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35479 } }
I20260808 02:16:57.865440 17593 leader_election.cc:302] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3120e47f03164c868ef852fdaa891fe1; no voters:
I20260808 02:16:57.865587 17593 leader_election.cc:288] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:57.865625 17599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:57.865780 17599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [term 1 LEADER]: Becoming Leader. State: Replica: 3120e47f03164c868ef852fdaa891fe1, State: Running, Role: LEADER
I20260808 02:16:57.865808 17593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:57.865865 17599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3120e47f03164c868ef852fdaa891fe1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35479 } }
I20260808 02:16:57.866364 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 17498
I20260808 02:16:57.866484 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:57.866675 19691 external_mini_cluster.cc:1428] Setting key 3703a8f41de7bebe6be29b4d4fa7aa55
I20260808 02:16:57.867101 17605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3120e47f03164c868ef852fdaa891fe1. Latest consensus state: current_term: 1 leader_uuid: "3120e47f03164c868ef852fdaa891fe1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3120e47f03164c868ef852fdaa891fe1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35479 } } }
I20260808 02:16:57.867199 17605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:57.867509 17602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3120e47f03164c868ef852fdaa891fe1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3120e47f03164c868ef852fdaa891fe1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35479 } } }
I20260808 02:16:57.867623 17602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1 [sys.catalog]: This master's current role is: LEADER
W20260808 02:16:57.867810 17623 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:16:57.867869 17623 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:16:57.867923 17627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:57.868104 17627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:57.869113 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.867080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57863 (local address 127.0.0.1:35479)
0808 02:16:57.867225 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:57.867233 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:57.867424 (+ 191us) server_negotiation.cc:407] Connection header received
0808 02:16:57.867630 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:57.867639 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:57.867669 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:57.867800 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:57.868675 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:57.868678 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:57.868678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:57.868682 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:57.868718 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:57.868728 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:57.868790 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:57.868879 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 02:16:57.868944 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260808 02:16:57.869962 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:16:57.872077 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:57.871161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43351 (local address 127.0.0.1:35479)
0808 02:16:57.871198 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:57.871200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:57.871264 (+ 64us) server_negotiation.cc:407] Connection header received
0808 02:16:57.871321 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:57.871322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:57.871326 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:57.871367 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:57.871886 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:57.871888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:57.871888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:16:57.871892 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:57.871906 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:16:57.871910 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:57.871957 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:57.872007 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 02:16:57.872020 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:16:57.876829 17627 catalog_manager.cc:1347] Generated new cluster ID: cea85582e5814904a71d151362fe8fd3
I20260808 02:16:57.876884 17627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:57.898568 17627 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:57.898730 17627 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:57.905514 17627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3120e47f03164c868ef852fdaa891fe1: Generated new TSK 0
I20260808 02:16:57.905680 17627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:57.917093 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 17498
2026-08-08T02:16:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:57 dist-test-slave-1l3l krb5kdc[17643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:57 dist-test-slave-1l3l krb5kdc[17643](info): set up 2 sockets
Aug 08 02:16:57 dist-test-slave-1l3l krb5kdc[17643](info): commencing operation
krb5kdc: starting...
W20260808 02:16:59.023393 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:16:59 dist-test-slave-1l3l krb5kdc[17643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:16:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:16:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:16:59.138139 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:16:59.218104 17661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:16:59.218294 17661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:16:59.218324 17661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:16:59.219467 17661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:16:59.219511 17661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:16:59.219524 17661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:16:59.219535 17661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:16:59.219547 17661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:16:59.221009 17661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:16:59.221223 17661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:16:59.221447 17661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:16:59.226222 17672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:59.226220 17674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:16:59.226231 17671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:16:59.226495 17661 server_base.cc:1034] running on GCE node
I20260808 02:16:59.226698 17661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:16:59.226961 17661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:16:59.228102 17661 hybrid_clock.cc:648] HybridClock initialized: now 1786155419228087 us; error 29 us; skew 500 ppm
Aug 08 02:16:59 dist-test-slave-1l3l krb5kdc[17643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155419, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:16:59.233210 17661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:16:59.233655 17661 webserver.cc:460] Webserver started at http://127.0.0.1:33021/ using document root <none> and password file <none>
I20260808 02:16:59.233855 17661 fs_manager.cc:359] Metadata directory not provided
I20260808 02:16:59.233902 17661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:16:59.234026 17661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:16:59.235008 17661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "5760b8c379d3489ab569628d9999906f"
format_stamp: "Formatted at 2026-08-08 02:16:59 on dist-test-slave-1l3l"
server_key: "a1645cafbdd054eda1d3c0c42108846b"
server_key_iv: "15d21d44f4b40d62c1edf684cba4d3f0"
server_key_version: "encryptionkey@0"
I20260808 02:16:59.235347 17661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "5760b8c379d3489ab569628d9999906f"
format_stamp: "Formatted at 2026-08-08 02:16:59 on dist-test-slave-1l3l"
server_key: "a1645cafbdd054eda1d3c0c42108846b"
server_key_iv: "15d21d44f4b40d62c1edf684cba4d3f0"
server_key_version: "encryptionkey@0"
I20260808 02:16:59.236642 17661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:16:59.237284 17684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:16:59.237448 17661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:16:59.237531 17661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "5760b8c379d3489ab569628d9999906f"
format_stamp: "Formatted at 2026-08-08 02:16:59 on dist-test-slave-1l3l"
server_key: "a1645cafbdd054eda1d3c0c42108846b"
server_key_iv: "15d21d44f4b40d62c1edf684cba4d3f0"
server_key_version: "encryptionkey@0"
I20260808 02:16:59.237591 17661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:16:59.252425 17661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:16:59.254364 17661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:16:59.254506 17661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:16:59.258525 17661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33205
I20260808 02:16:59.258562 17762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33205 every 8 connection(s)
I20260808 02:16:59.258867 17661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:16:59.259685 17763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:16:59.261862 17763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f: Bootstrap starting.
I20260808 02:16:59.262378 17763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f: Neither blocks nor log segments found. Creating new log.
I20260808 02:16:59.262431 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 17661
I20260808 02:16:59.262530 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:16:59.262661 17763 log.cc:824] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:16:59.262696 19691 external_mini_cluster.cc:1428] Setting key 8b4e768597fa7ec78bf9eaee0b22ae41
I20260808 02:16:59.263392 17763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f: No bootstrap required, opened a new log
I20260808 02:16:59.265493 17763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5760b8c379d3489ab569628d9999906f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } }
I20260808 02:16:59.265631 17763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:16:59.265666 17763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5760b8c379d3489ab569628d9999906f, State: Initialized, Role: FOLLOWER
I20260808 02:16:59.265784 17763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5760b8c379d3489ab569628d9999906f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } }
I20260808 02:16:59.265853 17763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:16:59.265883 17763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:16:59.265937 17763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:16:59.266469 17763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5760b8c379d3489ab569628d9999906f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } }
I20260808 02:16:59.266564 17763 leader_election.cc:302] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5760b8c379d3489ab569628d9999906f; no voters:
I20260808 02:16:59.266707 17763 leader_election.cc:288] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:16:59.266750 17771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:16:59.266955 17763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:16:59.266966 17771 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [term 1 LEADER]: Becoming Leader. State: Replica: 5760b8c379d3489ab569628d9999906f, State: Running, Role: LEADER
I20260808 02:16:59.267180 17771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5760b8c379d3489ab569628d9999906f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } }
W20260808 02:16:59.268801 17796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:16:59.268855 17796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 08 02:16:59 dist-test-slave-1l3l krb5kdc[17643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:16:59.269523 17773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5760b8c379d3489ab569628d9999906f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5760b8c379d3489ab569628d9999906f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } } }
I20260808 02:16:59.269616 17773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:59.269759 17774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5760b8c379d3489ab569628d9999906f. Latest consensus state: current_term: 1 leader_uuid: "5760b8c379d3489ab569628d9999906f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5760b8c379d3489ab569628d9999906f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33205 } } }
I20260808 02:16:59.269824 17798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:16:59.269883 17774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f [sys.catalog]: This master's current role is: LEADER
I20260808 02:16:59.270071 17798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:16:59.271564 17798 catalog_manager.cc:1347] Generated new cluster ID: a52e20c6e6cb4b2a9ca910bfcc4af1a1
I20260808 02:16:59.271612 17798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:16:59.273346 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:59.263203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:50377 (local address 127.0.0.1:33205)
0808 02:16:59.263379 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:59.263388 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:59.263420 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:16:59.263901 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:59.263908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:59.263955 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:59.264072 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:59.264674 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:59.265532 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:59.266389 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:59.266905 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:59.267013 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:59.269760 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:59.269764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:59.269765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:59.269777 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:59.271773 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:59.272206 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:59.272209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:59.272209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:59.272300 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:59.272522 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:59.272523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:59.272523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:59.272781 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:59.272930 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:59.273057 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 02:16:59.273155 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260808 02:16:59.274307 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:16:59.280489 17798 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:16:59.281004 17798 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:16:59.282845 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:16:59.275853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:38945 (local address 127.0.0.1:33205)
0808 02:16:59.275902 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:16:59.275903 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:16:59.276109 (+ 206us) server_negotiation.cc:407] Connection header received
0808 02:16:59.276137 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:16:59.276137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:16:59.276148 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:16:59.276214 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:16:59.276731 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:59.277456 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:59.278389 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:16:59.278723 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:16:59.278793 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:16:59.280094 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:16:59.280095 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:16:59.280096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:16:59.280103 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:16:59.281757 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:59.282132 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:59.282133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:59.282133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:59.282193 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:16:59.282381 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:16:59.282382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:16:59.282382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:16:59.282504 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:16:59.282604 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 02:16:59.282696 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 02:16:59.282758 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 02:16:59.298063 17798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5760b8c379d3489ab569628d9999906f: Generated new TSK 0
I20260808 02:16:59.298231 17798 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:16:59.301338 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 17661
W20260808 02:16:59.315881 17640 connection.cc:535] client connection to 127.0.0.1:33205 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33205): Connection reset by peer (error 104)
2026-08-08T02:16:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1384 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:16:59 dist-test-slave-1l3l krb5kdc[17812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:16:59 dist-test-slave-1l3l krb5kdc[17812](info): set up 2 sockets
Aug 08 02:16:59 dist-test-slave-1l3l krb5kdc[17812](info): commencing operation
krb5kdc: starting...
W20260808 02:17:00.423565 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:00 dist-test-slave-1l3l krb5kdc[17812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:00.535884 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45315
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:00.614310 17832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:00.614511 17832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:00.614539 17832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:00.615653 17832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:00.615694 17832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:00.615743 17832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:00.615764 17832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:00.615787 17832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:00.617268 17832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:00.617509 17832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:00.617712 17832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:00.622326 17838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:00.622457 17832 server_base.cc:1034] running on GCE node
W20260808 02:17:00.622402 17840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:00.622350 17837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:00.622809 17832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:00.623073 17832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:00.624224 17832 hybrid_clock.cc:648] HybridClock initialized: now 1786155420624184 us; error 61 us; skew 500 ppm
Aug 08 02:17:00 dist-test-slave-1l3l krb5kdc[17812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155420, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:00.626665 17832 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:00.627080 17832 webserver.cc:460] Webserver started at http://127.0.0.1:40079/ using document root <none> and password file <none>
I20260808 02:17:00.627274 17832 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:00.627321 17832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:00.627452 17832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:00.628535 17832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "90de953b359249b6bc7fa94ecce7fb34"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-1l3l"
server_key: "3be67a508dc10266ae5bc7d4f0f85e82"
server_key_iv: "9ae4801dce31bd8dbdf42f572a0f8d9b"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.628857 17832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "90de953b359249b6bc7fa94ecce7fb34"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-1l3l"
server_key: "3be67a508dc10266ae5bc7d4f0f85e82"
server_key_iv: "9ae4801dce31bd8dbdf42f572a0f8d9b"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.630105 17832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 02:17:00.630775 17848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:00.630935 17832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:17:00.631019 17832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "90de953b359249b6bc7fa94ecce7fb34"
format_stamp: "Formatted at 2026-08-08 02:17:00 on dist-test-slave-1l3l"
server_key: "3be67a508dc10266ae5bc7d4f0f85e82"
server_key_iv: "9ae4801dce31bd8dbdf42f572a0f8d9b"
server_key_version: "encryptionkey@0"
I20260808 02:17:00.631078 17832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:00.641520 17832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:00.643422 17832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:00.643555 17832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:00.647506 17832 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45315
I20260808 02:17:00.647547 17924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45315 every 8 connection(s)
I20260808 02:17:00.647938 17832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:00.648790 17925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:00.650097 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 17832
I20260808 02:17:00.650231 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:00.650424 19691 external_mini_cluster.cc:1428] Setting key 11cc507aa7eb284c8471edfedad274a8
I20260808 02:17:00.651379 17925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34: Bootstrap starting.
I20260808 02:17:00.651999 17925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:00.652323 17925 log.cc:824] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:00.653070 17925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34: No bootstrap required, opened a new log
I20260808 02:17:00.655167 17925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90de953b359249b6bc7fa94ecce7fb34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45315 } }
I20260808 02:17:00.655352 17925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:00.655393 17925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90de953b359249b6bc7fa94ecce7fb34, State: Initialized, Role: FOLLOWER
I20260808 02:17:00.655510 17925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90de953b359249b6bc7fa94ecce7fb34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45315 } }
I20260808 02:17:00.655593 17925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:00.655634 17925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:00.655679 17925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:00.656440 17925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90de953b359249b6bc7fa94ecce7fb34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45315 } }
I20260808 02:17:00.656546 17925 leader_election.cc:302] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90de953b359249b6bc7fa94ecce7fb34; no voters:
I20260808 02:17:00.656598 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.650997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:54289 (local address 127.0.0.1:45315)
0808 02:17:00.651187 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.651195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.651223 (+ 28us) server_negotiation.cc:407] Connection header received
0808 02:17:00.651760 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.651770 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.651829 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.651951 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:00.652182 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.653565 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.654367 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.654994 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.655108 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.655875 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:00.655879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:00.655879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:00.656083 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:00.656122 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:00.656136 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:00.656233 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.656369 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.656401 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260808 02:17:00.656698 17925 leader_election.cc:288] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:00.656877 17932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:00.657037 17932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [term 1 LEADER]: Becoming Leader. State: Replica: 90de953b359249b6bc7fa94ecce7fb34, State: Running, Role: LEADER
I20260808 02:17:00.657040 17925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:00.657198 17932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90de953b359249b6bc7fa94ecce7fb34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45315 } }
I20260808 02:17:00.657603 17931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90de953b359249b6bc7fa94ecce7fb34. Latest consensus state: current_term: 1 leader_uuid: "90de953b359249b6bc7fa94ecce7fb34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90de953b359249b6bc7fa94ecce7fb34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45315 } } }
I20260808 02:17:00.657644 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:00.657682 17931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:00.658779 17934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90de953b359249b6bc7fa94ecce7fb34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90de953b359249b6bc7fa94ecce7fb34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45315 } } }
W20260808 02:17:00.658962 17957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 02:17:00.659116 17934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34 [sys.catalog]: This master's current role is: LEADER
W20260808 02:17:00.659124 17957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:00.659200 17943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:00.659403 17943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:00.660883 17943 catalog_manager.cc:1347] Generated new cluster ID: 81da26f86a5447ebba1f6bcb229d201a
I20260808 02:17:00.660948 17943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:00.662758 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:00.659119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:46773 (local address 127.0.0.1:45315)
0808 02:17:00.659156 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:00.659158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:00.659871 (+ 713us) server_negotiation.cc:407] Connection header received
0808 02:17:00.659930 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:00.659931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:00.659943 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:00.659989 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:00.660447 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.660926 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.661646 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:00.661872 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:00.661926 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:00.662541 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:00.662543 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:00.662543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:00.662547 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:00.662561 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:00.662565 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:00.662609 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:00.662677 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 02:17:00.662685 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:17:00.684463 17943 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:00.685040 17943 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:00.693748 17943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90de953b359249b6bc7fa94ecce7fb34: Generated new TSK 0
I20260808 02:17:00.693931 17943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:00.703742 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 17832
W20260808 02:17:00.717908 17809 connection.cc:535] client connection to 127.0.0.1:45315 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45315): Connection reset by peer (error 104)
2026-08-08T02:17:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:00 dist-test-slave-1l3l krb5kdc[17973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:00 dist-test-slave-1l3l krb5kdc[17973](info): set up 2 sockets
Aug 08 02:17:00 dist-test-slave-1l3l krb5kdc[17973](info): commencing operation
krb5kdc: starting...
W20260808 02:17:01.824597 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:01 dist-test-slave-1l3l krb5kdc[17973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:01.939579 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46473
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:02.019553 17992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:02.019789 17992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:02.019812 17992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:02.020896 17992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:02.020927 17992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:02.020941 17992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:02.020952 17992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:02.020962 17992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:02.022336 17992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:02.022526 17992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:02.022702 17992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:02.027320 18001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:02.027325 18000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:02.027325 18003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:02.027554 17992 server_base.cc:1034] running on GCE node
I20260808 02:17:02.027848 17992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:02.028110 17992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:02.029251 17992 hybrid_clock.cc:648] HybridClock initialized: now 1786155422029239 us; error 28 us; skew 500 ppm
Aug 08 02:17:02 dist-test-slave-1l3l krb5kdc[17973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155422, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:02.031647 17992 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:02.032092 17992 webserver.cc:460] Webserver started at http://127.0.0.1:34581/ using document root <none> and password file <none>
I20260808 02:17:02.032294 17992 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:02.032342 17992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:02.032477 17992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:02.033461 17992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "df1a5eb98d214719a482d92257935f02"
format_stamp: "Formatted at 2026-08-08 02:17:02 on dist-test-slave-1l3l"
server_key: "25b6164c4378455e74ec6f1363edd00c"
server_key_iv: "87200a3fc5a2a9561152ddeee6d38bc9"
server_key_version: "encryptionkey@0"
I20260808 02:17:02.033779 17992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "df1a5eb98d214719a482d92257935f02"
format_stamp: "Formatted at 2026-08-08 02:17:02 on dist-test-slave-1l3l"
server_key: "25b6164c4378455e74ec6f1363edd00c"
server_key_iv: "87200a3fc5a2a9561152ddeee6d38bc9"
server_key_version: "encryptionkey@0"
I20260808 02:17:02.034956 17992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:17:02.035655 18016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:02.035828 17992 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:17:02.035908 17992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "df1a5eb98d214719a482d92257935f02"
format_stamp: "Formatted at 2026-08-08 02:17:02 on dist-test-slave-1l3l"
server_key: "25b6164c4378455e74ec6f1363edd00c"
server_key_iv: "87200a3fc5a2a9561152ddeee6d38bc9"
server_key_version: "encryptionkey@0"
I20260808 02:17:02.035969 17992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:02.039076 17992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:02.040757 17992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:02.040884 17992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:02.044790 17992 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46473
I20260808 02:17:02.044822 18089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46473 every 8 connection(s)
I20260808 02:17:02.045094 17992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:02.045976 18090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:02.048058 18090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02: Bootstrap starting.
I20260808 02:17:02.048545 18090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:02.048779 18090 log.cc:824] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:02.049352 18090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02: No bootstrap required, opened a new log
I20260808 02:17:02.050806 18090 raft_consensus.cc:348] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1a5eb98d214719a482d92257935f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46473 } }
I20260808 02:17:02.050937 18090 raft_consensus.cc:374] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:02.050966 18090 raft_consensus.cc:729] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1a5eb98d214719a482d92257935f02, State: Initialized, Role: FOLLOWER
I20260808 02:17:02.051060 18090 consensus_queue.cc:260] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1a5eb98d214719a482d92257935f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46473 } }
I20260808 02:17:02.051107 18090 raft_consensus.cc:388] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:02.051124 18090 raft_consensus.cc:482] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:02.051154 18090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:02.051834 18090 raft_consensus.cc:504] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1a5eb98d214719a482d92257935f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46473 } }
I20260808 02:17:02.051931 18090 leader_election.cc:302] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df1a5eb98d214719a482d92257935f02; no voters:
I20260808 02:17:02.052062 18090 leader_election.cc:288] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:02.052110 18094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:02.052263 18094 raft_consensus.cc:686] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [term 1 LEADER]: Becoming Leader. State: Replica: df1a5eb98d214719a482d92257935f02, State: Running, Role: LEADER
I20260808 02:17:02.052309 18090 sys_catalog.cc:564] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:02.052397 18094 consensus_queue.cc:237] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1a5eb98d214719a482d92257935f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46473 } }
I20260808 02:17:02.052749 18095 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df1a5eb98d214719a482d92257935f02. Latest consensus state: current_term: 1 leader_uuid: "df1a5eb98d214719a482d92257935f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1a5eb98d214719a482d92257935f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46473 } } }
I20260808 02:17:02.052875 18095 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:02.052863 18096 sys_catalog.cc:455] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df1a5eb98d214719a482d92257935f02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df1a5eb98d214719a482d92257935f02" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46473 } } }
I20260808 02:17:02.053172 18096 sys_catalog.cc:458] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:02.053334 18104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:02.053825 18104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:02.054342 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 17992
I20260808 02:17:02.054491 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:02.054669 19691 external_mini_cluster.cc:1428] Setting key 0f9c3c6669526f745ec6453949c7fa26
I20260808 02:17:02.055226 18104 catalog_manager.cc:1347] Generated new cluster ID: a5496888e32c400dadc16d86b4a42b4c
I20260808 02:17:02.055276 18104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:02.057001 18120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:02.055064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:52207 (local address 127.0.0.1:46473)
0808 02:17:02.055581 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:02.055590 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:02.055617 (+ 27us) server_negotiation.cc:407] Connection header received
0808 02:17:02.055743 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:02.055749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:02.055777 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:02.055893 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:02.056596 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:02.056600 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:02.056600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:02.056604 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:02.056643 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:02.056655 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:02.056712 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:02.056785 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 02:17:02.056823 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":73,"threads_started":1}
I20260808 02:17:02.057757 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:02.064733 18120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:02.059955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:41701 (local address 127.0.0.1:46473)
0808 02:17:02.059987 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:02.059988 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:02.063894 (+ 3906us) server_negotiation.cc:407] Connection header received
0808 02:17:02.063958 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:02.063959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:02.063964 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:02.064015 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:02.064547 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:02.064548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:02.064549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:02.064552 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:02.064565 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:02.064569 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:02.064610 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:02.064660 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 02:17:02.064672 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 02:17:02.070802 18104 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:02.070947 18104 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:02.075261 18104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df1a5eb98d214719a482d92257935f02: Generated new TSK 0
I20260808 02:17:02.075424 18104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:02.080700 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 17992
2026-08-08T02:17:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1377 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:02 dist-test-slave-1l3l krb5kdc[18134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:02 dist-test-slave-1l3l krb5kdc[18134](info): set up 2 sockets
Aug 08 02:17:02 dist-test-slave-1l3l krb5kdc[18134](info): commencing operation
krb5kdc: starting...
W20260808 02:17:03.209781 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:03 dist-test-slave-1l3l krb5kdc[18134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:03.320461 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45023
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:03.399102 18153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:03.399290 18153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:03.399313 18153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:03.400431 18153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:03.400472 18153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:03.400485 18153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:03.400496 18153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:03.400516 18153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:03.401944 18153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45023
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:03.402151 18153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:03.402348 18153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:03.406797 18159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:03.406838 18164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:03.406805 18161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:03.406834 18153 server_base.cc:1034] running on GCE node
I20260808 02:17:03.407210 18153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:03.407475 18153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:03.408614 18153 hybrid_clock.cc:648] HybridClock initialized: now 1786155423408594 us; error 36 us; skew 500 ppm
Aug 08 02:17:03 dist-test-slave-1l3l krb5kdc[18134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155423, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:03.410916 18153 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:03.411281 18153 webserver.cc:460] Webserver started at http://127.0.0.1:46177/ using document root <none> and password file <none>
I20260808 02:17:03.411442 18153 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:03.411480 18153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:03.411567 18153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:03.412515 18153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "16059f029e7148258b69e3b3600c27c7"
format_stamp: "Formatted at 2026-08-08 02:17:03 on dist-test-slave-1l3l"
server_key: "46192af952a5edacd6b6c41b152fad1c"
server_key_iv: "7dfffda1337a367cb1ecff9dba49e83c"
server_key_version: "encryptionkey@0"
I20260808 02:17:03.412797 18153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "16059f029e7148258b69e3b3600c27c7"
format_stamp: "Formatted at 2026-08-08 02:17:03 on dist-test-slave-1l3l"
server_key: "46192af952a5edacd6b6c41b152fad1c"
server_key_iv: "7dfffda1337a367cb1ecff9dba49e83c"
server_key_version: "encryptionkey@0"
I20260808 02:17:03.413982 18153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:17:03.414585 18174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:03.414740 18153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:17:03.414832 18153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "16059f029e7148258b69e3b3600c27c7"
format_stamp: "Formatted at 2026-08-08 02:17:03 on dist-test-slave-1l3l"
server_key: "46192af952a5edacd6b6c41b152fad1c"
server_key_iv: "7dfffda1337a367cb1ecff9dba49e83c"
server_key_version: "encryptionkey@0"
I20260808 02:17:03.414885 18153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:03.433606 18153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:03.435371 18153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:03.435479 18153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:03.439288 18153 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45023
I20260808 02:17:03.439335 18245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45023 every 8 connection(s)
I20260808 02:17:03.439623 18153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:03.440507 18247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:03.442747 18247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7: Bootstrap starting.
I20260808 02:17:03.443305 18247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:03.443562 18247 log.cc:824] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:03.444231 18247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7: No bootstrap required, opened a new log
I20260808 02:17:03.444480 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 18153
I20260808 02:17:03.444597 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:03.444769 19691 external_mini_cluster.cc:1428] Setting key 6c3300d3788fc786fc9cee313f058736
I20260808 02:17:03.446336 18247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16059f029e7148258b69e3b3600c27c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260808 02:17:03.446540 18247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:03.446578 18247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16059f029e7148258b69e3b3600c27c7, State: Initialized, Role: FOLLOWER
I20260808 02:17:03.446707 18247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16059f029e7148258b69e3b3600c27c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260808 02:17:03.446784 18247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:03.446823 18247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:03.446867 18247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:03.447615 18247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16059f029e7148258b69e3b3600c27c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260808 02:17:03.447726 18247 leader_election.cc:302] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16059f029e7148258b69e3b3600c27c7; no voters:
I20260808 02:17:03.447877 18247 leader_election.cc:288] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:03.447916 18256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:03.448081 18256 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [term 1 LEADER]: Becoming Leader. State: Replica: 16059f029e7148258b69e3b3600c27c7, State: Running, Role: LEADER
I20260808 02:17:03.448134 18247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:03.448223 18256 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16059f029e7148258b69e3b3600c27c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } }
I20260808 02:17:03.448764 18259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16059f029e7148258b69e3b3600c27c7. Latest consensus state: current_term: 1 leader_uuid: "16059f029e7148258b69e3b3600c27c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16059f029e7148258b69e3b3600c27c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } } }
I20260808 02:17:03.448855 18259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:03.448752 18260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16059f029e7148258b69e3b3600c27c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16059f029e7148258b69e3b3600c27c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45023 } } }
I20260808 02:17:03.448951 18260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:03.449311 18274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:03.449796 18274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:03.451283 18274 catalog_manager.cc:1347] Generated new cluster ID: bd411bf99cd34c588e37740ba49b2600
I20260808 02:17:03.451332 18274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:17:03 dist-test-slave-1l3l krb5kdc[18134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 02:17:03.455546 18253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:03.445279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:44225 (local address 127.0.0.1:45023)
0808 02:17:03.445506 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:03.445513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:03.445538 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:17:03.445994 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:03.446001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:03.446097 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:03.446222 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:03.446960 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:03.447942 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:03.449011 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:03.449619 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:03.449731 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:03.452341 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:03.452345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:03.452346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:03.452362 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:03.453963 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:03.454314 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:03.454317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:03.454317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:03.454401 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:03.454605 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:03.454606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:03.454606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:03.454835 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:03.454987 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:03.455106 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 02:17:03.455203 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":127,"threads_started":1}
I20260808 02:17:03.456418 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:03.463182 18253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:03.457633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:60573 (local address 127.0.0.1:45023)
0808 02:17:03.457664 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:03.457666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:03.457809 (+ 143us) server_negotiation.cc:407] Connection header received
0808 02:17:03.457874 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:03.457875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:03.457884 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:03.457923 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:03.458391 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:03.458870 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:03.459521 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:03.459799 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:03.459861 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:03.460936 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:03.460938 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:03.460938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:03.460944 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:03.462162 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:03.462494 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:03.462495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:03.462495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:03.462546 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:03.462732 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:03.462733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:03.462733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:03.462840 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:03.462929 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:03.463044 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 02:17:03.463102 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 02:17:03.464004 18274 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:03.464483 18274 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:03.475531 18274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16059f029e7148258b69e3b3600c27c7: Generated new TSK 0
I20260808 02:17:03.475677 18274 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:03.482035 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 18153
W20260808 02:17:03.492969 18129 connection.cc:535] client connection to 127.0.0.1:45023 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45023): Connection reset by peer (error 104)
2026-08-08T02:17:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:03 dist-test-slave-1l3l krb5kdc[18300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:03 dist-test-slave-1l3l krb5kdc[18300](info): set up 2 sockets
Aug 08 02:17:03 dist-test-slave-1l3l krb5kdc[18300](info): commencing operation
krb5kdc: starting...
W20260808 02:17:04.588361 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:04 dist-test-slave-1l3l krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:04.700186 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35613
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:04.779172 18319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:04.779378 18319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:04.779408 18319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:04.780561 18319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:04.780608 18319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:04.780628 18319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:04.780645 18319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:04.780659 18319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:04.782102 18319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35613
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:04.782333 18319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:04.782536 18319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 02:17:04.787299 18319 server_base.cc:1034] running on GCE node
W20260808 02:17:04.787253 18327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:04.787256 18325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:04.787295 18324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:04.787797 18319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:04.788095 18319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:04.789222 18319 hybrid_clock.cc:648] HybridClock initialized: now 1786155424789207 us; error 30 us; skew 500 ppm
Aug 08 02:17:04 dist-test-slave-1l3l krb5kdc[18300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155424, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:04.794313 18319 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:04.794697 18319 webserver.cc:460] Webserver started at http://127.0.0.1:41591/ using document root <none> and password file <none>
I20260808 02:17:04.794863 18319 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:04.794909 18319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:04.794997 18319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:04.796026 18319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "ed07b6220b2f420e8e121a89b4e5ca93"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-1l3l"
server_key: "17dbfdfcd117be63edf8da9b56694004"
server_key_iv: "574adc58f5bda0644928bdbe8aa511b8"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.796317 18319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "ed07b6220b2f420e8e121a89b4e5ca93"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-1l3l"
server_key: "17dbfdfcd117be63edf8da9b56694004"
server_key_iv: "574adc58f5bda0644928bdbe8aa511b8"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.797511 18319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:17:04.798149 18334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:04.798293 18319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:17:04.798370 18319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "ed07b6220b2f420e8e121a89b4e5ca93"
format_stamp: "Formatted at 2026-08-08 02:17:04 on dist-test-slave-1l3l"
server_key: "17dbfdfcd117be63edf8da9b56694004"
server_key_iv: "574adc58f5bda0644928bdbe8aa511b8"
server_key_version: "encryptionkey@0"
I20260808 02:17:04.798452 18319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:04.831387 18319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:04.833539 18319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:04.833650 18319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:04.837445 18319 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35613
I20260808 02:17:04.837754 18407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35613 every 8 connection(s)
I20260808 02:17:04.837795 18319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:04.838413 18408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:04.840678 18408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93: Bootstrap starting.
I20260808 02:17:04.841233 18408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:04.841511 18408 log.cc:824] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:04.842139 18408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93: No bootstrap required, opened a new log
I20260808 02:17:04.843689 18408 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07b6220b2f420e8e121a89b4e5ca93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35613 } }
I20260808 02:17:04.843851 18408 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:04.843873 18408 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed07b6220b2f420e8e121a89b4e5ca93, State: Initialized, Role: FOLLOWER
I20260808 02:17:04.844010 18408 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07b6220b2f420e8e121a89b4e5ca93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35613 } }
I20260808 02:17:04.844085 18408 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:04.844120 18408 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:04.844164 18408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:04.844504 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 18319
I20260808 02:17:04.844630 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:04.844825 19691 external_mini_cluster.cc:1428] Setting key 3df1d7d6fb3d9449c7d2f0b17c436a2e
I20260808 02:17:04.844822 18408 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07b6220b2f420e8e121a89b4e5ca93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35613 } }
I20260808 02:17:04.844918 18408 leader_election.cc:302] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed07b6220b2f420e8e121a89b4e5ca93; no voters:
I20260808 02:17:04.845074 18408 leader_election.cc:288] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:04.845111 18413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:04.845280 18413 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [term 1 LEADER]: Becoming Leader. State: Replica: ed07b6220b2f420e8e121a89b4e5ca93, State: Running, Role: LEADER
I20260808 02:17:04.845358 18408 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:04.845366 18413 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07b6220b2f420e8e121a89b4e5ca93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35613 } }
I20260808 02:17:04.845757 18413 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed07b6220b2f420e8e121a89b4e5ca93. Latest consensus state: current_term: 1 leader_uuid: "ed07b6220b2f420e8e121a89b4e5ca93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07b6220b2f420e8e121a89b4e5ca93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35613 } } }
I20260808 02:17:04.845824 18413 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:04.846174 18414 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed07b6220b2f420e8e121a89b4e5ca93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed07b6220b2f420e8e121a89b4e5ca93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35613 } } }
I20260808 02:17:04.846289 18414 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:04.846489 18425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:04.846987 18425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:04.848518 18425 catalog_manager.cc:1347] Generated new cluster ID: 0c11df594e2e408b9cf148b5adca97dd
I20260808 02:17:04.848568 18425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:04.850854 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.845548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:38347 (local address 127.0.0.1:35613)
0808 02:17:04.845787 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.845796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.845827 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:17:04.845940 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.845947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.846003 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.846398 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:04.846520 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.847919 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.848833 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.849442 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.849546 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.850201 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:04.850205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:04.850206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:04.850420 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:04.850457 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:04.850472 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:04.850557 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.850621 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.850648 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260808 02:17:04.851933 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:04.856729 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:04.853373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:49045 (local address 127.0.0.1:35613)
0808 02:17:04.853417 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:04.853418 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:04.853539 (+ 121us) server_negotiation.cc:407] Connection header received
0808 02:17:04.853602 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:04.853603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:04.853614 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:04.853662 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:04.853770 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.854461 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.855492 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:04.855807 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:04.855861 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:04.856499 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:04.856500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:04.856501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:04.856505 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:04.856518 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:04.856523 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:04.856581 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:04.856648 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 02:17:04.856658 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:17:04.857707 18425 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:04.858389 18425 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:04.865691 18425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed07b6220b2f420e8e121a89b4e5ca93: Generated new TSK 0
I20260808 02:17:04.865842 18425 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:04.867383 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 18319
W20260808 02:17:04.880160 18295 connection.cc:535] client connection to 127.0.0.1:35613 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35613): Connection reset by peer (error 104)
2026-08-08T02:17:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1387 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:04 dist-test-slave-1l3l krb5kdc[18459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:04 dist-test-slave-1l3l krb5kdc[18459](info): set up 2 sockets
Aug 08 02:17:04 dist-test-slave-1l3l krb5kdc[18459](info): commencing operation
krb5kdc: starting...
W20260808 02:17:05.976462 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:06 dist-test-slave-1l3l krb5kdc[18459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 02:17:06.090541 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35031
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:06.172163 18477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:06.172374 18477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:06.172395 18477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:06.173491 18477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:06.173525 18477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:06.173537 18477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:06.173548 18477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:06.173559 18477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:06.175015 18477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35031
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:06.175200 18477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:06.175397 18477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:06.179941 18482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:06.179942 18488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:06.179929 18484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:06.180049 18477 server_base.cc:1034] running on GCE node
I20260808 02:17:06.180354 18477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:06.180609 18477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:06.181748 18477 hybrid_clock.cc:648] HybridClock initialized: now 1786155426181732 us; error 28 us; skew 500 ppm
Aug 08 02:17:06 dist-test-slave-1l3l krb5kdc[18459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155426, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:06.184109 18477 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 02:17:06.184497 18477 webserver.cc:460] Webserver started at http://127.0.0.1:40027/ using document root <none> and password file <none>
I20260808 02:17:06.184695 18477 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:06.184738 18477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:06.184854 18477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:06.185801 18477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "670d6be1b0ae450e9f14b25867b29e74"
format_stamp: "Formatted at 2026-08-08 02:17:06 on dist-test-slave-1l3l"
server_key: "ed2d429db54f6d789b9a88bbc3d90070"
server_key_iv: "fa670863a636b85d3b062d6fb6e742eb"
server_key_version: "encryptionkey@0"
I20260808 02:17:06.186122 18477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "670d6be1b0ae450e9f14b25867b29e74"
format_stamp: "Formatted at 2026-08-08 02:17:06 on dist-test-slave-1l3l"
server_key: "ed2d429db54f6d789b9a88bbc3d90070"
server_key_iv: "fa670863a636b85d3b062d6fb6e742eb"
server_key_version: "encryptionkey@0"
I20260808 02:17:06.187343 18477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 02:17:06.187994 18501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:06.188117 18477 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:06.188191 18477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "670d6be1b0ae450e9f14b25867b29e74"
format_stamp: "Formatted at 2026-08-08 02:17:06 on dist-test-slave-1l3l"
server_key: "ed2d429db54f6d789b9a88bbc3d90070"
server_key_iv: "fa670863a636b85d3b062d6fb6e742eb"
server_key_version: "encryptionkey@0"
I20260808 02:17:06.188252 18477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:06.191425 18477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:06.193039 18477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:06.193151 18477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:06.196936 18477 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35031
I20260808 02:17:06.196977 18579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35031 every 8 connection(s)
I20260808 02:17:06.197239 18477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:06.198119 18580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:06.200224 18580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74: Bootstrap starting.
I20260808 02:17:06.200721 18580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:06.200954 18580 log.cc:824] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:06.201557 18580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74: No bootstrap required, opened a new log
I20260808 02:17:06.203029 18580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d6be1b0ae450e9f14b25867b29e74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35031 } }
I20260808 02:17:06.203150 18580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:06.203169 18580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670d6be1b0ae450e9f14b25867b29e74, State: Initialized, Role: FOLLOWER
I20260808 02:17:06.203264 18580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d6be1b0ae450e9f14b25867b29e74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35031 } }
I20260808 02:17:06.203310 18580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:06.203332 18580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:06.203361 18580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:06.203874 18580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d6be1b0ae450e9f14b25867b29e74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35031 } }
I20260808 02:17:06.203958 18580 leader_election.cc:302] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 670d6be1b0ae450e9f14b25867b29e74; no voters:
I20260808 02:17:06.204078 18580 leader_election.cc:288] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:06.204126 18584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:06.204301 18584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [term 1 LEADER]: Becoming Leader. State: Replica: 670d6be1b0ae450e9f14b25867b29e74, State: Running, Role: LEADER
I20260808 02:17:06.204286 18580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:06.204443 18584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d6be1b0ae450e9f14b25867b29e74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35031 } }
I20260808 02:17:06.204784 18587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 670d6be1b0ae450e9f14b25867b29e74. Latest consensus state: current_term: 1 leader_uuid: "670d6be1b0ae450e9f14b25867b29e74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d6be1b0ae450e9f14b25867b29e74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35031 } } }
I20260808 02:17:06.204793 18585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "670d6be1b0ae450e9f14b25867b29e74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670d6be1b0ae450e9f14b25867b29e74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35031 } } }
I20260808 02:17:06.204854 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 18477
I20260808 02:17:06.204876 18587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:06.204888 18585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:06.204975 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:06.205075 18594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:06.205138 19691 external_mini_cluster.cc:1428] Setting key c70768b79f654752b1b0a291e9f32a5a
I20260808 02:17:06.205626 18594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:06.207180 18594 catalog_manager.cc:1347] Generated new cluster ID: 97697550e7dd4cc8ab2870b434308710
I20260808 02:17:06.207233 18594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:06.208177 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:06.205739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:38473 (local address 127.0.0.1:35031)
0808 02:17:06.206578 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:06.206586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:06.206619 (+ 33us) server_negotiation.cc:407] Connection header received
0808 02:17:06.206728 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:06.206735 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:06.206765 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:06.206906 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:06.207784 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:06.207788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:06.207788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:06.207792 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:06.207826 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:06.207837 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:06.207888 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:06.207967 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 02:17:06.208011 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":792,"thread_start_us":63,"threads_started":1}
I20260808 02:17:06.209020 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:06.211172 18616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:06.210252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.111:36971 (local address 127.0.0.1:35031)
0808 02:17:06.210282 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:06.210284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:06.210403 (+ 119us) server_negotiation.cc:407] Connection header received
0808 02:17:06.210440 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:06.210441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:06.210445 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:06.210486 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:06.210984 (+ 498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:06.210985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:06.210986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:06.210990 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:06.211002 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:06.211006 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:06.211052 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:06.211102 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 02:17:06.211115 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 02:17:06.218060 18594 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:06.218199 18594 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:06.222509 18594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 670d6be1b0ae450e9f14b25867b29e74: Generated new TSK 0
I20260808 02:17:06.222668 18594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:06.229242 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 18477
2026-08-08T02:17:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:06 dist-test-slave-1l3l krb5kdc[18631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:06 dist-test-slave-1l3l krb5kdc[18631](info): set up 2 sockets
Aug 08 02:17:06 dist-test-slave-1l3l krb5kdc[18631](info): commencing operation
krb5kdc: starting...
W20260808 02:17:07.357752 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:07 dist-test-slave-1l3l krb5kdc[18631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:17:07.470763 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:38739
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:39443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:38739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:07.550499 18651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:07.550756 18651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:07.550789 18651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:07.551981 18651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:07.552024 18651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:07.552043 18651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:07.552059 18651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:07.552086 18651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:07.553530 18651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:38739
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:38739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:07.553753 18651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:07.553964 18651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:07.558634 18661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:07.558652 18659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:07.558638 18664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:07.558821 18651 server_base.cc:1034] running on GCE node
I20260808 02:17:07.559067 18651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:07.559345 18651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:07.560504 18651 hybrid_clock.cc:648] HybridClock initialized: now 1786155427560488 us; error 28 us; skew 500 ppm
Aug 08 02:17:07 dist-test-slave-1l3l krb5kdc[18631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155427, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:07.563050 18651 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:17:07.563513 18651 webserver.cc:460] Webserver started at http://127.19.58.254:46563/ using document root <none> and password file <none>
I20260808 02:17:07.563768 18651 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:07.563820 18651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:07.563936 18651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:07.564992 18651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "dbf1d0dc359343e7a5d81df983a8266a"
format_stamp: "Formatted at 2026-08-08 02:17:07 on dist-test-slave-1l3l"
server_key: "33936552544904590f6863e18b8e2ed5"
server_key_iv: "3b8bbea66edfa27c94fe8ccc117282fe"
server_key_version: "encryptionkey@0"
I20260808 02:17:07.565323 18651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "dbf1d0dc359343e7a5d81df983a8266a"
format_stamp: "Formatted at 2026-08-08 02:17:07 on dist-test-slave-1l3l"
server_key: "33936552544904590f6863e18b8e2ed5"
server_key_iv: "3b8bbea66edfa27c94fe8ccc117282fe"
server_key_version: "encryptionkey@0"
I20260808 02:17:07.566576 18651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:17:07.567276 18671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:07.567471 18651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:17:07.567559 18651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "dbf1d0dc359343e7a5d81df983a8266a"
format_stamp: "Formatted at 2026-08-08 02:17:07 on dist-test-slave-1l3l"
server_key: "33936552544904590f6863e18b8e2ed5"
server_key_iv: "3b8bbea66edfa27c94fe8ccc117282fe"
server_key_version: "encryptionkey@0"
I20260808 02:17:07.567627 18651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:07.583319 18651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:07.585414 18651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:07.585551 18651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:07.589464 18651 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:38739
I20260808 02:17:07.589506 18748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:38739 every 8 connection(s)
I20260808 02:17:07.589836 18651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:07.590659 18749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:07.592971 18749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a: Bootstrap starting.
I20260808 02:17:07.593531 18749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:07.593804 18749 log.cc:824] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:07.594515 18749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a: No bootstrap required, opened a new log
I20260808 02:17:07.595193 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 18651
I20260808 02:17:07.595315 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:07.595525 19691 external_mini_cluster.cc:1428] Setting key 19b94f787e632e73254249cba1a404ff
I20260808 02:17:07.596590 18749 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1d0dc359343e7a5d81df983a8266a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38739 } }
I20260808 02:17:07.596732 18749 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:07.596769 18749 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf1d0dc359343e7a5d81df983a8266a, State: Initialized, Role: FOLLOWER
I20260808 02:17:07.596956 18749 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1d0dc359343e7a5d81df983a8266a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38739 } }
I20260808 02:17:07.597043 18749 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:07.597074 18749 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:07.597121 18749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:07.598059 18749 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1d0dc359343e7a5d81df983a8266a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38739 } }
I20260808 02:17:07.598160 18749 leader_election.cc:302] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbf1d0dc359343e7a5d81df983a8266a; no voters:
I20260808 02:17:07.598306 18749 leader_election.cc:288] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:07.598539 18749 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:07.598659 18760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:07.598820 18760 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [term 1 LEADER]: Becoming Leader. State: Replica: dbf1d0dc359343e7a5d81df983a8266a, State: Running, Role: LEADER
I20260808 02:17:07.598940 18760 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1d0dc359343e7a5d81df983a8266a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38739 } }
I20260808 02:17:07.599321 18760 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbf1d0dc359343e7a5d81df983a8266a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1d0dc359343e7a5d81df983a8266a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38739 } } }
I20260808 02:17:07.599450 18760 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:07.599615 18760 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbf1d0dc359343e7a5d81df983a8266a. Latest consensus state: current_term: 1 leader_uuid: "dbf1d0dc359343e7a5d81df983a8266a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf1d0dc359343e7a5d81df983a8266a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 38739 } } }
I20260808 02:17:07.599762 18760 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:07.599920 18775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:07.600569 18775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 02:17:07 dist-test-slave-1l3l krb5kdc[18631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:17:07.607177 18775 catalog_manager.cc:1347] Generated new cluster ID: 001568387a7e49ff8074376efba9e5ba
I20260808 02:17:07.607227 18775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:07.607656 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:07.596051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56523 (local address 127.19.58.254:38739)
0808 02:17:07.596250 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:07.596257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:07.596283 (+ 26us) server_negotiation.cc:407] Connection header received
0808 02:17:07.596563 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:07.596569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:07.596612 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:07.596718 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:07.597497 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:07.598504 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:07.599538 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:07.600359 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:07.600460 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:07.603628 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:07.603633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:07.603633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:07.603645 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:07.605738 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:07.606164 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:07.606167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:07.606168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:07.606300 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:07.606555 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:07.606572 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:07.606572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:07.606910 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:07.607093 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:07.607233 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 02:17:07.607457 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260808 02:17:07.608620 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:07.620126 18775 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:07.620661 18775 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:07.624303 18775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbf1d0dc359343e7a5d81df983a8266a: Generated new TSK 0
I20260808 02:17:07.624464 18775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:07.627089 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:07.619977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44267 (local address 127.19.58.254:38739)
0808 02:17:07.620020 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:07.620022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:07.620111 (+ 89us) server_negotiation.cc:407] Connection header received
0808 02:17:07.620236 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:07.620237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:07.620268 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:07.620329 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:07.620936 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:07.621648 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:07.622642 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:07.623313 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:07.623383 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:07.624785 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:07.624786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:07.624786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:07.624792 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:07.626054 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:07.626452 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:07.626452 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:07.626453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:07.626501 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:07.626688 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:07.626688 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:07.626688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:07.626775 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:07.626843 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:07.626981 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 02:17:07.627029 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:17:07.628993 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 18651
W20260808 02:17:07.643452 18628 connection.cc:535] client connection to 127.19.58.254:38739 recv error: Network error: failed to read from TLS socket (remote: 127.19.58.254:38739): Connection reset by peer (error 104)
2026-08-08T02:17:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:07 dist-test-slave-1l3l krb5kdc[18798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:07 dist-test-slave-1l3l krb5kdc[18798](info): set up 2 sockets
Aug 08 02:17:07 dist-test-slave-1l3l krb5kdc[18798](info): commencing operation
krb5kdc: starting...
W20260808 02:17:08.746611 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:08 dist-test-slave-1l3l krb5kdc[18798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:17:08.855733 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:45813
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:45813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:08.933835 18819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:08.934026 18819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:08.934057 18819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:08.935160 18819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:08.935202 18819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:08.935220 18819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:08.935235 18819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:08.935251 18819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:08.936734 18819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:45813
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:45813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:08.936969 18819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:08.937170 18819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:08.941558 18831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:08.941583 18827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:08.941568 18826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:08.941632 18819 server_base.cc:1034] running on GCE node
I20260808 02:17:08.942022 18819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:08.942283 18819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:08.943430 18819 hybrid_clock.cc:648] HybridClock initialized: now 1786155428943416 us; error 27 us; skew 500 ppm
Aug 08 02:17:08 dist-test-slave-1l3l krb5kdc[18798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155428, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:08.945709 18819 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:17:08.946086 18819 webserver.cc:460] Webserver started at http://127.19.58.254:45541/ using document root <none> and password file <none>
I20260808 02:17:08.946291 18819 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:08.946337 18819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:08.946467 18819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:08.947461 18819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "5e327b1891a64e88a31a0d9d642a8e5b"
format_stamp: "Formatted at 2026-08-08 02:17:08 on dist-test-slave-1l3l"
server_key: "0e3243bd0e1354aacc7504afcea664db"
server_key_iv: "f303e6c5de24eefa566910b2677aed36"
server_key_version: "encryptionkey@0"
I20260808 02:17:08.947835 18819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "5e327b1891a64e88a31a0d9d642a8e5b"
format_stamp: "Formatted at 2026-08-08 02:17:08 on dist-test-slave-1l3l"
server_key: "0e3243bd0e1354aacc7504afcea664db"
server_key_iv: "f303e6c5de24eefa566910b2677aed36"
server_key_version: "encryptionkey@0"
I20260808 02:17:08.949082 18819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:17:08.949805 18842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:08.949944 18819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:17:08.950024 18819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "5e327b1891a64e88a31a0d9d642a8e5b"
format_stamp: "Formatted at 2026-08-08 02:17:08 on dist-test-slave-1l3l"
server_key: "0e3243bd0e1354aacc7504afcea664db"
server_key_iv: "f303e6c5de24eefa566910b2677aed36"
server_key_version: "encryptionkey@0"
I20260808 02:17:08.950088 18819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:08.971287 18819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:08.973030 18819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:08.973158 18819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:08.978667 18819 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:45813
I20260808 02:17:08.978700 18925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:45813 every 8 connection(s)
I20260808 02:17:08.979001 18819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:08.979791 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 18819
I20260808 02:17:08.979907 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:08.979916 18926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:08.980113 19691 external_mini_cluster.cc:1428] Setting key 2418699724397e80e65f2e85e48c4ef1
I20260808 02:17:08.982690 18926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b: Bootstrap starting.
I20260808 02:17:08.983374 18926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:08.983675 18926 log.cc:824] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:08.984390 18926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b: No bootstrap required, opened a new log
I20260808 02:17:08.985373 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:08.980612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50459 (local address 127.19.58.254:45813)
0808 02:17:08.980761 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:08.980768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:08.980792 (+ 24us) server_negotiation.cc:407] Connection header received
0808 02:17:08.981219 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:08.981226 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:08.981270 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:08.981373 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:08.981600 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:08.982466 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:08.983511 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:08.984063 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:08.984166 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:08.984786 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:08.984790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:08.984790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:08.984943 (+ 153us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:08.984970 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:08.984980 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:08.985043 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:08.985144 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 02:17:08.985166 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260808 02:17:08.985918 18926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e327b1891a64e88a31a0d9d642a8e5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45813 } }
I20260808 02:17:08.986044 18926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:08.986076 18926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e327b1891a64e88a31a0d9d642a8e5b, State: Initialized, Role: FOLLOWER
I20260808 02:17:08.986210 18926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e327b1891a64e88a31a0d9d642a8e5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45813 } }
I20260808 02:17:08.986282 18926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:08.986313 18926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:08.986361 18926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:08.986507 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:08.987015 18926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e327b1891a64e88a31a0d9d642a8e5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45813 } }
I20260808 02:17:08.987125 18926 leader_election.cc:302] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e327b1891a64e88a31a0d9d642a8e5b; no voters:
I20260808 02:17:08.987304 18926 leader_election.cc:288] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:08.987371 18934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:08.987547 18926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:08.987612 18934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [term 1 LEADER]: Becoming Leader. State: Replica: 5e327b1891a64e88a31a0d9d642a8e5b, State: Running, Role: LEADER
I20260808 02:17:08.987783 18934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e327b1891a64e88a31a0d9d642a8e5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45813 } }
I20260808 02:17:08.988118 18935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e327b1891a64e88a31a0d9d642a8e5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e327b1891a64e88a31a0d9d642a8e5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45813 } } }
I20260808 02:17:08.988193 18935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:08.988332 18935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e327b1891a64e88a31a0d9d642a8e5b. Latest consensus state: current_term: 1 leader_uuid: "5e327b1891a64e88a31a0d9d642a8e5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e327b1891a64e88a31a0d9d642a8e5b" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 45813 } } }
I20260808 02:17:08.988379 18935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:08.988399 18951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:08.988901 18951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:08.990393 18951 catalog_manager.cc:1347] Generated new cluster ID: 246015b6b7e046ab86d760c592890263
I20260808 02:17:08.990446 18951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:08.991885 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:08.988936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47405 (local address 127.19.58.254:45813)
0808 02:17:08.988971 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:08.988973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:08.989099 (+ 126us) server_negotiation.cc:407] Connection header received
0808 02:17:08.989205 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:08.989206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:08.989216 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:08.989270 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:08.989411 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:08.990045 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:08.990820 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:08.991025 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:08.991078 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:08.991653 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:08.991654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:08.991654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:08.991658 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:08.991671 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:08.991675 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:08.991752 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:08.991808 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 02:17:08.991817 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:17:08.997151 18951 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:08.997627 18951 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:09.008114 18951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e327b1891a64e88a31a0d9d642a8e5b: Generated new TSK 0
I20260808 02:17:09.008260 18951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:09.012562 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 18819
W20260808 02:17:09.029951 18796 connection.cc:535] client connection to 127.19.58.254:45813 recv error: Network error: failed to read from TLS socket (remote: 127.19.58.254:45813): Connection reset by peer (error 104)
2026-08-08T02:17:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1386 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:09 dist-test-slave-1l3l krb5kdc[18968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:09 dist-test-slave-1l3l krb5kdc[18968](info): set up 2 sockets
Aug 08 02:17:09 dist-test-slave-1l3l krb5kdc[18968](info): commencing operation
krb5kdc: starting...
W20260808 02:17:10.140718 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:10 dist-test-slave-1l3l krb5kdc[18968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:17:10.255458 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:41185
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:39485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:41185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:10.334074 18987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:10.334273 18987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:10.334295 18987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:10.335377 18987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:10.335413 18987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:10.335425 18987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:10.335436 18987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:10.335448 18987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:10.336840 18987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:39485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:41185
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:41185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:10.337033 18987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:10.337234 18987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:10.341744 18993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:10.341754 18994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:10.341754 18996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:10.342015 18987 server_base.cc:1034] running on GCE node
I20260808 02:17:10.342222 18987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:10.342473 18987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:10.343604 18987 hybrid_clock.cc:648] HybridClock initialized: now 1786155430343590 us; error 29 us; skew 500 ppm
Aug 08 02:17:10 dist-test-slave-1l3l krb5kdc[18968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155430, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:10.350180 18987 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:17:10.350618 18987 webserver.cc:460] Webserver started at http://127.19.58.254:32879/ using document root <none> and password file <none>
I20260808 02:17:10.350811 18987 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:10.350855 18987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:10.350986 18987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:10.352077 18987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "159c9415417e45f6bd1c2c770967f74a"
format_stamp: "Formatted at 2026-08-08 02:17:10 on dist-test-slave-1l3l"
server_key: "6b61ca78a68379908b4300e30148ff97"
server_key_iv: "1ebb0f33507eb7d26ad274d75a4a5ee8"
server_key_version: "encryptionkey@0"
I20260808 02:17:10.352407 18987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "159c9415417e45f6bd1c2c770967f74a"
format_stamp: "Formatted at 2026-08-08 02:17:10 on dist-test-slave-1l3l"
server_key: "6b61ca78a68379908b4300e30148ff97"
server_key_iv: "1ebb0f33507eb7d26ad274d75a4a5ee8"
server_key_version: "encryptionkey@0"
I20260808 02:17:10.353633 18987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:17:10.354262 19007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:10.354414 18987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:17:10.354501 18987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "159c9415417e45f6bd1c2c770967f74a"
format_stamp: "Formatted at 2026-08-08 02:17:10 on dist-test-slave-1l3l"
server_key: "6b61ca78a68379908b4300e30148ff97"
server_key_iv: "1ebb0f33507eb7d26ad274d75a4a5ee8"
server_key_version: "encryptionkey@0"
I20260808 02:17:10.354559 18987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:10.357710 18987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:10.359665 18987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:10.359828 18987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:10.363469 18987 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:41185
I20260808 02:17:10.363499 19079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:41185 every 8 connection(s)
I20260808 02:17:10.363814 18987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:10.364763 19080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:10.366894 19080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a: Bootstrap starting.
I20260808 02:17:10.367425 19080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:10.367686 19080 log.cc:824] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:10.368361 19080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a: No bootstrap required, opened a new log
I20260808 02:17:10.369611 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 18987
I20260808 02:17:10.369709 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:10.369864 19691 external_mini_cluster.cc:1428] Setting key 414be0528ca953baa1692ac92b62d5bd
I20260808 02:17:10.369975 19080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159c9415417e45f6bd1c2c770967f74a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41185 } }
I20260808 02:17:10.370108 19080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:10.370141 19080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 159c9415417e45f6bd1c2c770967f74a, State: Initialized, Role: FOLLOWER
I20260808 02:17:10.370275 19080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159c9415417e45f6bd1c2c770967f74a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41185 } }
I20260808 02:17:10.370395 19080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:10.370445 19080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:10.370543 19080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:10.371474 19080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159c9415417e45f6bd1c2c770967f74a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41185 } }
I20260808 02:17:10.371572 19080 leader_election.cc:302] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 159c9415417e45f6bd1c2c770967f74a; no voters:
I20260808 02:17:10.371757 19080 leader_election.cc:288] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:10.371781 19088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:10.371963 19088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [term 1 LEADER]: Becoming Leader. State: Replica: 159c9415417e45f6bd1c2c770967f74a, State: Running, Role: LEADER
I20260808 02:17:10.371982 19085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:10.370367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54213 (local address 127.19.58.254:41185)
0808 02:17:10.370557 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:10.370567 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:10.370592 (+ 25us) server_negotiation.cc:407] Connection header received
0808 02:17:10.370700 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:10.370706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:10.370728 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:10.370841 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:10.371500 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:10.371504 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:10.371504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:10.371508 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:10.371551 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:10.371563 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:10.371625 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:10.371746 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 02:17:10.371792 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260808 02:17:10.372066 19080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:10.372128 19088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159c9415417e45f6bd1c2c770967f74a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41185 } }
I20260808 02:17:10.373073 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:10.373643 19093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 159c9415417e45f6bd1c2c770967f74a. Latest consensus state: current_term: 1 leader_uuid: "159c9415417e45f6bd1c2c770967f74a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159c9415417e45f6bd1c2c770967f74a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41185 } } }
I20260808 02:17:10.373782 19090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "159c9415417e45f6bd1c2c770967f74a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159c9415417e45f6bd1c2c770967f74a" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41185 } } }
W20260808 02:17:10.373798 19110 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 02:17:10.374606 19093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:10.374620 19090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a [sys.catalog]: This master's current role is: LEADER
W20260808 02:17:10.374631 19110 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:10.375228 19118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:10.375480 19118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:10.375798 19085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:10.374508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47305 (local address 127.19.58.254:41185)
0808 02:17:10.374541 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:10.374543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:10.374741 (+ 198us) server_negotiation.cc:407] Connection header received
0808 02:17:10.374835 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:10.374836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:10.374852 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:10.374879 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:10.375512 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:10.375514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:10.375514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:10.375518 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:10.375532 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:10.375536 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:10.375591 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:10.375685 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 02:17:10.375699 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:17:10.377019 19118 catalog_manager.cc:1347] Generated new cluster ID: 7c638988a97d42f2bf061b2ab8886926
I20260808 02:17:10.377071 19118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:10.391652 19118 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:10.391835 19118 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:10.404323 19118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 159c9415417e45f6bd1c2c770967f74a: Generated new TSK 0
I20260808 02:17:10.404524 19118 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:10.407563 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 18987
2026-08-08T02:17:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:10 dist-test-slave-1l3l krb5kdc[19124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:10 dist-test-slave-1l3l krb5kdc[19124](info): set up 2 sockets
Aug 08 02:17:10 dist-test-slave-1l3l krb5kdc[19124](info): commencing operation
krb5kdc: starting...
W20260808 02:17:11.522665 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:11 dist-test-slave-1l3l krb5kdc[19124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:17:11.636030 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42837
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:42837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:11.714929 19147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:11.715122 19147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:11.715152 19147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:11.716267 19147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:11.716308 19147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:11.716327 19147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:11.716343 19147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:11.716357 19147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:11.717787 19147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:42837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:42837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:11.718020 19147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:11.718222 19147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:11.722677 19158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:11.722729 19155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:11.722862 19147 server_base.cc:1034] running on GCE node
W20260808 02:17:11.722895 19153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:11.723116 19147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:11.723389 19147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:11.724532 19147 hybrid_clock.cc:648] HybridClock initialized: now 1786155431724516 us; error 30 us; skew 500 ppm
Aug 08 02:17:11 dist-test-slave-1l3l krb5kdc[19124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155431, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:11.726866 19147 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:17:11.727283 19147 webserver.cc:460] Webserver started at http://127.19.58.254:33795/ using document root <none> and password file <none>
I20260808 02:17:11.727468 19147 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:11.727515 19147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:11.727634 19147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:11.728628 19147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "ebf2a7ecdcfb40f39e569aa358d823fa"
format_stamp: "Formatted at 2026-08-08 02:17:11 on dist-test-slave-1l3l"
server_key: "e6b861731f229bee04eee64edc912202"
server_key_iv: "e28ae30c3f34d769d6e55af39372a8ba"
server_key_version: "encryptionkey@0"
I20260808 02:17:11.728947 19147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "ebf2a7ecdcfb40f39e569aa358d823fa"
format_stamp: "Formatted at 2026-08-08 02:17:11 on dist-test-slave-1l3l"
server_key: "e6b861731f229bee04eee64edc912202"
server_key_iv: "e28ae30c3f34d769d6e55af39372a8ba"
server_key_version: "encryptionkey@0"
I20260808 02:17:11.730134 19147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:17:11.730777 19167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:11.730937 19147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:17:11.731016 19147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "ebf2a7ecdcfb40f39e569aa358d823fa"
format_stamp: "Formatted at 2026-08-08 02:17:11 on dist-test-slave-1l3l"
server_key: "e6b861731f229bee04eee64edc912202"
server_key_iv: "e28ae30c3f34d769d6e55af39372a8ba"
server_key_version: "encryptionkey@0"
I20260808 02:17:11.731076 19147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:11.747959 19147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:11.749612 19147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:11.749742 19147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:11.753630 19147 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:42837
I20260808 02:17:11.753662 19257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:42837 every 8 connection(s)
I20260808 02:17:11.753974 19147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:11.754773 19258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:11.756959 19258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa: Bootstrap starting.
I20260808 02:17:11.757493 19258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:11.757742 19258 log.cc:824] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:11.758366 19258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa: No bootstrap required, opened a new log
I20260808 02:17:11.759974 19258 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf2a7ecdcfb40f39e569aa358d823fa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42837 } }
I20260808 02:17:11.760138 19258 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:11.760183 19258 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebf2a7ecdcfb40f39e569aa358d823fa, State: Initialized, Role: FOLLOWER
I20260808 02:17:11.760227 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 19147
I20260808 02:17:11.760335 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:11.760334 19258 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf2a7ecdcfb40f39e569aa358d823fa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42837 } }
I20260808 02:17:11.760433 19258 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:11.760471 19258 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:11.760516 19258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:11.760522 19691 external_mini_cluster.cc:1428] Setting key cc924b593508b1c42ec4cc64f6bb0828
I20260808 02:17:11.761409 19258 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf2a7ecdcfb40f39e569aa358d823fa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42837 } }
I20260808 02:17:11.761519 19258 leader_election.cc:302] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebf2a7ecdcfb40f39e569aa358d823fa; no voters:
I20260808 02:17:11.761682 19258 leader_election.cc:288] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:11.761703 19263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:11.761839 19263 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [term 1 LEADER]: Becoming Leader. State: Replica: ebf2a7ecdcfb40f39e569aa358d823fa, State: Running, Role: LEADER
I20260808 02:17:11.761924 19258 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:11.761978 19263 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf2a7ecdcfb40f39e569aa358d823fa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42837 } }
I20260808 02:17:11.762286 19265 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebf2a7ecdcfb40f39e569aa358d823fa. Latest consensus state: current_term: 1 leader_uuid: "ebf2a7ecdcfb40f39e569aa358d823fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf2a7ecdcfb40f39e569aa358d823fa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42837 } } }
I20260808 02:17:11.762378 19265 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:11.762591 19271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:11.763103 19271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:11.763866 19264 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebf2a7ecdcfb40f39e569aa358d823fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf2a7ecdcfb40f39e569aa358d823fa" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 42837 } } }
I20260808 02:17:11.763942 19264 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:11.764802 19271 catalog_manager.cc:1347] Generated new cluster ID: e1a417ca9e4b4b958d9ebe4da2e75ac7
I20260808 02:17:11.764854 19271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:17:11 dist-test-slave-1l3l krb5kdc[19124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:17:11.771399 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:11.760962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57243 (local address 127.19.58.254:42837)
0808 02:17:11.761125 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:11.761133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:11.761179 (+ 46us) server_negotiation.cc:407] Connection header received
0808 02:17:11.761264 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:11.761269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:11.761309 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:11.761404 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:11.762223 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:11.763586 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:11.764637 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:11.765230 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:11.765326 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:11.767999 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:11.768003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:11.768003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:11.768015 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:11.769856 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:11.770214 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:11.770218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:11.770218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:11.770345 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:11.770530 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:11.770531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:11.770532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:11.770838 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:11.771004 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:11.771069 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 02:17:11.771200 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260808 02:17:11.772302 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:11.772627 19271 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:11.773085 19271 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:11.780337 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:11.773566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35791 (local address 127.19.58.254:42837)
0808 02:17:11.773597 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:11.773599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:11.773842 (+ 243us) server_negotiation.cc:407] Connection header received
0808 02:17:11.773884 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:11.773885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:11.773897 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:11.773949 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:11.774497 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:11.775235 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:11.776240 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:11.776558 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:11.776626 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:11.777838 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:11.777839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:11.777839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:11.777846 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:11.779285 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:11.779616 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:11.779617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:11.779618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:11.779672 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:11.779867 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:11.779868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:11.779868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:11.779997 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:11.780080 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:11.780177 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 02:17:11.780250 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":17}
I20260808 02:17:11.786396 19271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebf2a7ecdcfb40f39e569aa358d823fa: Generated new TSK 0
I20260808 02:17:11.786553 19271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:11.790858 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 19147
2026-08-08T02:17:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:11 dist-test-slave-1l3l krb5kdc[19301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:11 dist-test-slave-1l3l krb5kdc[19301](info): set up 2 sockets
Aug 08 02:17:11 dist-test-slave-1l3l krb5kdc[19301](info): commencing operation
krb5kdc: starting...
W20260808 02:17:12.924585 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:13 dist-test-slave-1l3l krb5kdc[19301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:17:13.038866 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:34957
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:35579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:34957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:13.116902 19322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:13.117089 19322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:13.117120 19322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:13.118202 19322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:13.118242 19322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:13.118260 19322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:13.118276 19322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:13.118292 19322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:13.119673 19322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:35579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:34957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:34957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:13.119916 19322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:13.120108 19322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:13.124498 19333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:13.124550 19330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:13.124576 19331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:13.124828 19322 server_base.cc:1034] running on GCE node
I20260808 02:17:13.125018 19322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:13.125245 19322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:13.126380 19322 hybrid_clock.cc:648] HybridClock initialized: now 1786155433126365 us; error 35 us; skew 500 ppm
Aug 08 02:17:13 dist-test-slave-1l3l krb5kdc[19301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155433, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:13.128763 19322 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:17:13.129165 19322 webserver.cc:460] Webserver started at http://127.19.58.254:45729/ using document root <none> and password file <none>
I20260808 02:17:13.129343 19322 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:13.129390 19322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:13.129511 19322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:13.130455 19322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "54175b4de8004084bac53b5f1285608f"
format_stamp: "Formatted at 2026-08-08 02:17:13 on dist-test-slave-1l3l"
server_key: "0e59926041b0833ffcebf6549d7d430d"
server_key_iv: "9a29e17f9a27e548e7e4e8affbc7a430"
server_key_version: "encryptionkey@0"
I20260808 02:17:13.130765 19322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "54175b4de8004084bac53b5f1285608f"
format_stamp: "Formatted at 2026-08-08 02:17:13 on dist-test-slave-1l3l"
server_key: "0e59926041b0833ffcebf6549d7d430d"
server_key_iv: "9a29e17f9a27e548e7e4e8affbc7a430"
server_key_version: "encryptionkey@0"
I20260808 02:17:13.131968 19322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:17:13.132633 19345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:13.132789 19322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:17:13.132867 19322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "54175b4de8004084bac53b5f1285608f"
format_stamp: "Formatted at 2026-08-08 02:17:13 on dist-test-slave-1l3l"
server_key: "0e59926041b0833ffcebf6549d7d430d"
server_key_iv: "9a29e17f9a27e548e7e4e8affbc7a430"
server_key_version: "encryptionkey@0"
I20260808 02:17:13.132936 19322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:13.141808 19322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:13.143559 19322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:13.143688 19322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:13.147325 19322 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:34957
I20260808 02:17:13.147364 19427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:34957 every 8 connection(s)
I20260808 02:17:13.147648 19322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:13.148459 19428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:13.150521 19428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f: Bootstrap starting.
I20260808 02:17:13.151032 19428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:13.151286 19428 log.cc:824] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:13.151942 19428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f: No bootstrap required, opened a new log
I20260808 02:17:13.152873 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 19322
I20260808 02:17:13.152977 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:13.153133 19691 external_mini_cluster.cc:1428] Setting key 2473b84a6b9aa915d6c1dc7eb7576927
I20260808 02:17:13.153746 19428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54175b4de8004084bac53b5f1285608f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34957 } }
I20260808 02:17:13.153910 19428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:13.153954 19428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54175b4de8004084bac53b5f1285608f, State: Initialized, Role: FOLLOWER
I20260808 02:17:13.154071 19428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54175b4de8004084bac53b5f1285608f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34957 } }
I20260808 02:17:13.154152 19428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:13.154194 19428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:13.154237 19428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:13.154884 19428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54175b4de8004084bac53b5f1285608f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34957 } }
I20260808 02:17:13.154989 19428 leader_election.cc:302] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54175b4de8004084bac53b5f1285608f; no voters:
I20260808 02:17:13.155140 19428 leader_election.cc:288] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:13.155176 19433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:13.155318 19433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [term 1 LEADER]: Becoming Leader. State: Replica: 54175b4de8004084bac53b5f1285608f, State: Running, Role: LEADER
I20260808 02:17:13.155368 19428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:13.155407 19433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54175b4de8004084bac53b5f1285608f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34957 } }
I20260808 02:17:13.155752 19438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54175b4de8004084bac53b5f1285608f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54175b4de8004084bac53b5f1285608f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34957 } } }
I20260808 02:17:13.155851 19438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:13.155745 19433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54175b4de8004084bac53b5f1285608f. Latest consensus state: current_term: 1 leader_uuid: "54175b4de8004084bac53b5f1285608f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54175b4de8004084bac53b5f1285608f" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 34957 } } }
I20260808 02:17:13.155956 19433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:13.156960 19448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:13.157495 19448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:13.158972 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:13.153565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54043 (local address 127.19.58.254:34957)
0808 02:17:13.153756 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:13.153765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:13.153797 (+ 32us) server_negotiation.cc:407] Connection header received
0808 02:17:13.153925 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:13.153933 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:13.153988 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:13.154125 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:13.154451 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:13.155857 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:13.156640 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:13.157228 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:13.157632 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:13.158209 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:13.158213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:13.158214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:13.158391 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:13.158422 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:13.158434 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:13.158494 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:13.158687 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 02:17:13.158750 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260808 02:17:13.160084 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:13.160344 19448 catalog_manager.cc:1347] Generated new cluster ID: e1d1ab81fd8643d383320a775d654c55
I20260808 02:17:13.160384 19448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:13.174281 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:13.168030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47171 (local address 127.19.58.254:34957)
0808 02:17:13.168066 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:13.168068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:13.171864 (+ 3796us) server_negotiation.cc:407] Connection header received
0808 02:17:13.171934 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:13.171935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:13.171958 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:13.172006 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:13.172119 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:13.172595 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:13.173257 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:13.173451 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:13.173506 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:13.174066 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:13.174067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:13.174067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:13.174073 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:13.174086 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:13.174089 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:13.174134 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:13.174189 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 02:17:13.174211 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 02:17:13.176396 19448 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:13.176844 19448 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:13.182000 19448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54175b4de8004084bac53b5f1285608f: Generated new TSK 0
I20260808 02:17:13.182137 19448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:13.185449 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 19322
2026-08-08T02:17:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:13 dist-test-slave-1l3l krb5kdc[19468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:13 dist-test-slave-1l3l krb5kdc[19468](info): set up 2 sockets
Aug 08 02:17:13 dist-test-slave-1l3l krb5kdc[19468](info): commencing operation
krb5kdc: starting...
W20260808 02:17:14.306660 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:14 dist-test-slave-1l3l krb5kdc[19468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:17:14.418848 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40795
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:37229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:40795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:14.499248 19486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:14.499464 19486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:14.499495 19486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:14.500648 19486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:14.500689 19486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:14.500708 19486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:14.500725 19486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:14.500741 19486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:14.502194 19486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:37229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:40795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:14.502429 19486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:14.502633 19486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:14.507272 19498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:14.507278 19495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:14.507297 19500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:14.507542 19486 server_base.cc:1034] running on GCE node
I20260808 02:17:14.507789 19486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:14.508103 19486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:14.509253 19486 hybrid_clock.cc:648] HybridClock initialized: now 1786155434509237 us; error 32 us; skew 500 ppm
Aug 08 02:17:14 dist-test-slave-1l3l krb5kdc[19468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155434, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:14.511597 19486 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:17:14.512053 19486 webserver.cc:460] Webserver started at http://127.19.58.254:45933/ using document root <none> and password file <none>
I20260808 02:17:14.512259 19486 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:14.512308 19486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:14.512430 19486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:14.513437 19486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "206c66bb3f824efba70895b8e39b8b63"
format_stamp: "Formatted at 2026-08-08 02:17:14 on dist-test-slave-1l3l"
server_key: "bf9b681188aef2b3dc62dee46d0d31c0"
server_key_iv: "747ffaa65c4843090b87a9d2e824352b"
server_key_version: "encryptionkey@0"
I20260808 02:17:14.513767 19486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "206c66bb3f824efba70895b8e39b8b63"
format_stamp: "Formatted at 2026-08-08 02:17:14 on dist-test-slave-1l3l"
server_key: "bf9b681188aef2b3dc62dee46d0d31c0"
server_key_iv: "747ffaa65c4843090b87a9d2e824352b"
server_key_version: "encryptionkey@0"
I20260808 02:17:14.514993 19486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 02:17:14.515641 19507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:14.515803 19486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:17:14.515884 19486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "206c66bb3f824efba70895b8e39b8b63"
format_stamp: "Formatted at 2026-08-08 02:17:14 on dist-test-slave-1l3l"
server_key: "bf9b681188aef2b3dc62dee46d0d31c0"
server_key_iv: "747ffaa65c4843090b87a9d2e824352b"
server_key_version: "encryptionkey@0"
I20260808 02:17:14.515957 19486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:14.519044 19486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:14.520743 19486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:14.520870 19486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:14.524672 19486 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:40795
I20260808 02:17:14.524703 19605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:40795 every 8 connection(s)
I20260808 02:17:14.524986 19486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:14.525846 19606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:14.527959 19606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63: Bootstrap starting.
I20260808 02:17:14.528453 19606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:14.528676 19606 log.cc:824] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:14.529268 19606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63: No bootstrap required, opened a new log
I20260808 02:17:14.530790 19606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c66bb3f824efba70895b8e39b8b63" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40795 } }
I20260808 02:17:14.530930 19606 raft_consensus.cc:374] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 02:17:14.530953 19606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 206c66bb3f824efba70895b8e39b8b63, State: Initialized, Role: FOLLOWER
I20260808 02:17:14.531057 19606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c66bb3f824efba70895b8e39b8b63" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40795 } }
I20260808 02:17:14.531131 19606 raft_consensus.cc:388] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 02:17:14.531158 19606 raft_consensus.cc:482] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 02:17:14.531188 19606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:14.531988 19606 raft_consensus.cc:504] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c66bb3f824efba70895b8e39b8b63" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40795 } }
I20260808 02:17:14.532073 19606 leader_election.cc:302] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 206c66bb3f824efba70895b8e39b8b63; no voters:
I20260808 02:17:14.532182 19606 leader_election.cc:288] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 02:17:14.532223 19611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:14.532366 19611 raft_consensus.cc:686] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [term 1 LEADER]: Becoming Leader. State: Replica: 206c66bb3f824efba70895b8e39b8b63, State: Running, Role: LEADER
I20260808 02:17:14.532406 19606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:14.532459 19611 consensus_queue.cc:237] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c66bb3f824efba70895b8e39b8b63" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40795 } }
I20260808 02:17:14.532857 19615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 206c66bb3f824efba70895b8e39b8b63. Latest consensus state: current_term: 1 leader_uuid: "206c66bb3f824efba70895b8e39b8b63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c66bb3f824efba70895b8e39b8b63" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40795 } } }
I20260808 02:17:14.532950 19615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:14.532840 19614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "206c66bb3f824efba70895b8e39b8b63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206c66bb3f824efba70895b8e39b8b63" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40795 } } }
I20260808 02:17:14.533116 19614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:14.533167 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 19486
I20260808 02:17:14.533253 19628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:14.533282 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:14.533444 19691 external_mini_cluster.cc:1428] Setting key 95b1423ba284d899f648f4ce47271bea
I20260808 02:17:14.533834 19628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:14.535436 19628 catalog_manager.cc:1347] Generated new cluster ID: 8405ed72cf2c42a3b3822d9fde316b0c
I20260808 02:17:14.535487 19628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:14.535899 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:14.534071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43307 (local address 127.19.58.254:40795)
0808 02:17:14.534571 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:14.534578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:14.534601 (+ 23us) server_negotiation.cc:407] Connection header received
0808 02:17:14.534683 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:14.534688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:14.534710 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:14.534820 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:14.535475 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:14.535479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:14.535479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:14.535484 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:14.535527 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:14.535541 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:14.535611 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:14.535678 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 02:17:14.535736 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":70,"threads_started":1}
I20260808 02:17:14.536778 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:14.544298 19628 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:14.544421 19628 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:14.544896 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:14.540087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60441 (local address 127.19.58.254:40795)
0808 02:17:14.540122 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:14.540123 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:14.543868 (+ 3745us) server_negotiation.cc:407] Connection header received
0808 02:17:14.543955 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:14.543955 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:14.543961 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:14.544013 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:14.544696 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:14.544697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:14.544698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 02:17:14.544702 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:14.544717 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 02:17:14.544722 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:14.544779 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:14.544823 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 02:17:14.544835 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:17:14.548739 19628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 206c66bb3f824efba70895b8e39b8b63: Generated new TSK 0
I20260808 02:17:14.548885 19628 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 02:17:14.551195 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 19486
2026-08-08T02:17:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1357 ms)
[----------] 18 tests from AuthTokenIssuingTest (24965 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:14 dist-test-slave-1l3l krb5kdc[19657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:14 dist-test-slave-1l3l krb5kdc[19657](info): set up 2 sockets
Aug 08 02:17:14 dist-test-slave-1l3l krb5kdc[19657](info): commencing operation
krb5kdc: starting...
W20260808 02:17:15.669055 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:15 dist-test-slave-1l3l krb5kdc[19657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:17:15.782121 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40555
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:40555,127.19.58.253:43433,127.19.58.252:36155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:15.861070 19681 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:15.861270 19681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:15.861295 19681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:15.861320 19681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:15.862398 19681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:15.862438 19681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:15.862457 19681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:15.862473 19681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:15.862488 19681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:15.863966 19681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:40555,127.19.58.253:43433,127.19.58.252:36155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:40555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:15.864208 19681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:15.864421 19681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:15.868899 19701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:15.868913 19697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:15.868909 19690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:15.869071 19681 server_base.cc:1034] running on GCE node
I20260808 02:17:15.869349 19681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:15.869606 19681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:15.870738 19681 hybrid_clock.cc:648] HybridClock initialized: now 1786155435870719 us; error 37 us; skew 500 ppm
Aug 08 02:17:15 dist-test-slave-1l3l krb5kdc[19657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:15.878654 19681 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:17:15.879029 19681 webserver.cc:460] Webserver started at http://127.19.58.254:36259/ using document root <none> and password file <none>
I20260808 02:17:15.879202 19681 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:15.879241 19681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:15.879331 19681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:15.880364 19681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "a8b22e76871440dbbba2decb4e903140"
format_stamp: "Formatted at 2026-08-08 02:17:15 on dist-test-slave-1l3l"
server_key: "16b90a63a54a3b4336bd71bc6574ba11"
server_key_iv: "fe1f12f85a7a689dbb3b281ff4f18724"
server_key_version: "encryptionkey@0"
I20260808 02:17:15.880661 19681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "a8b22e76871440dbbba2decb4e903140"
format_stamp: "Formatted at 2026-08-08 02:17:15 on dist-test-slave-1l3l"
server_key: "16b90a63a54a3b4336bd71bc6574ba11"
server_key_iv: "fe1f12f85a7a689dbb3b281ff4f18724"
server_key_version: "encryptionkey@0"
I20260808 02:17:15.881814 19681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:17:15.882432 19711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:15.882592 19681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:17:15.882678 19681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "a8b22e76871440dbbba2decb4e903140"
format_stamp: "Formatted at 2026-08-08 02:17:15 on dist-test-slave-1l3l"
server_key: "16b90a63a54a3b4336bd71bc6574ba11"
server_key_iv: "fe1f12f85a7a689dbb3b281ff4f18724"
server_key_version: "encryptionkey@0"
I20260808 02:17:15.882745 19681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:15.894310 19681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:15.896123 19681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:15.896253 19681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:15.900017 19681 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:40555
I20260808 02:17:15.900064 19791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:40555 every 8 connection(s)
I20260808 02:17:15.900352 19681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:15.901257 19793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:15.902674 19793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:15.906461 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 19681
I20260808 02:17:15.906546 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:15.906698 19691 external_mini_cluster.cc:1428] Setting key 3c9320498f6011691c975b964f5e903b
Aug 08 02:17:15 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:17:15.911993 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:15.903284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.19.58.254:40555)
0808 02:17:15.903436 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:15.903445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:15.903553 (+ 108us) server_negotiation.cc:407] Connection header received
0808 02:17:15.904146 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:15.904148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:15.904164 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:15.904238 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:15.904751 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:15.905564 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:15.906377 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:15.906705 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:15.906816 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:15.908995 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:15.909003 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:15.909003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:15.909011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:15.910481 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:15.910922 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:15.910924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:15.910925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:15.910993 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:15.911231 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:15.911232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:15.911232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:15.911431 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:15.911489 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:15.911728 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 02:17:15.911828 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260808 02:17:15.912063 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:15.903243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40555 (local address 127.0.0.1:52300)
0808 02:17:15.903426 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:15.903468 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:15.904000 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:15.904252 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:15.904254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:15.904514 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:15.904659 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:15.904664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:15.905648 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:15.905649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:15.906299 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:15.906301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:15.906816 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:15.906818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:15.906904 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:15.907352 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:15.907362 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:15.908912 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:15.910562 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:15.910570 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:15.910572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:15.910844 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:15.911057 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:15.911058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:15.911058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:15.911167 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:15.911495 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:15.911501 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:15.911574 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 02:17:15.911675 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 02:17:15.911928 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":77,"thread_start_us":91,"threads_started":1}
I20260808 02:17:15.912711 19793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:15.913038 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:15.912918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:43433 (local address 127.0.0.1:53934)
0808 02:17:15.912964 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:15.913012 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 02:17:15.913178 19716 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.58.253:43433: connect: Connection refused (error 111)
I20260808 02:17:15.913587 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:15.913488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:43433 (local address 127.0.0.1:53946)
0808 02:17:15.913532 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:15.913564 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 02:17:15.913688 19793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.253:43433: Network error: Client connection negotiation failed: client connection to 127.19.58.253:43433: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.19.58.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab.
Entry for principal kudu/127.19.58.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab.
I20260808 02:17:15.930881 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.253:43433
--webserver_interface=127.19.58.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab
--principal=kudu/127.19.58.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:40555,127.19.58.253:43433,127.19.58.252:36155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:17:15.962917 19793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } attempt: 1
I20260808 02:17:15.963415 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:15.963270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:43433 (local address 127.0.0.1:53954)
0808 02:17:15.963336 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:15.963381 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 02:17:15.963783 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:15.963666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:43433 (local address 127.0.0.1:53964)
0808 02:17:15.963730 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:15.963759 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 02:17:15.963876 19793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.253:43433: Network error: Client connection negotiation failed: client connection to 127.19.58.253:43433: connect: Connection refused (error 111)
W20260808 02:17:16.009979 19803 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:16.010180 19803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:16.010197 19803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:16.010215 19803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:16.011318 19803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:16.011361 19803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:16.011375 19803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:16.011399 19803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:16.011415 19803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:16.013197 19803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:40555,127.19.58.253:43433,127.19.58.252:36155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab
--principal=kudu/127.19.58.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.253:43433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.58.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:16.013432 19803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:16.013651 19803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:16.018182 19813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:16.018151 19815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:16.018158 19811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:16.018340 19803 server_base.cc:1034] running on GCE node
I20260808 02:17:16.018632 19803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:16.018896 19803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:16.020037 19803 hybrid_clock.cc:648] HybridClock initialized: now 1786155436020023 us; error 29 us; skew 500 ppm
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:16.022398 19803 init.cc:377] Logged in from keytab as kudu/127.19.58.253@KRBTEST.COM (short username kudu)
I20260808 02:17:16.022780 19803 webserver.cc:460] Webserver started at http://127.19.58.253:46855/ using document root <none> and password file <none>
I20260808 02:17:16.022984 19803 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:16.023025 19803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:16.023114 19803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:16.024185 19803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/data/instance:
uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc"
format_stamp: "Formatted at 2026-08-08 02:17:16 on dist-test-slave-1l3l"
server_key: "fb301a21b7de59aaad4265bc4a0a5a9b"
server_key_iv: "e56ec827b8ba326b44cd78ec4058d867"
server_key_version: "encryptionkey@0"
I20260808 02:17:16.024485 19803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/wal/instance:
uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc"
format_stamp: "Formatted at 2026-08-08 02:17:16 on dist-test-slave-1l3l"
server_key: "fb301a21b7de59aaad4265bc4a0a5a9b"
server_key_iv: "e56ec827b8ba326b44cd78ec4058d867"
server_key_version: "encryptionkey@0"
I20260808 02:17:16.025736 19803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:17:16.026436 19826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:16.026607 19803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:17:16.026690 19803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/wal
uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc"
format_stamp: "Formatted at 2026-08-08 02:17:16 on dist-test-slave-1l3l"
server_key: "fb301a21b7de59aaad4265bc4a0a5a9b"
server_key_iv: "e56ec827b8ba326b44cd78ec4058d867"
server_key_version: "encryptionkey@0"
I20260808 02:17:16.026749 19803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:16.032022 19793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } attempt: 2
I20260808 02:17:16.032398 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.032273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:43433 (local address 127.0.0.1:53968)
0808 02:17:16.032327 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.032366 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 02:17:16.032689 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.032612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:43433 (local address 127.0.0.1:53976)
0808 02:17:16.032642 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.032671 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:43433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 02:17:16.032764 19793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.253:43433: Network error: Client connection negotiation failed: client connection to 127.19.58.253:43433: connect: Connection refused (error 111)
I20260808 02:17:16.040287 19803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:16.042335 19803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:16.042471 19803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:16.046307 19803 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.253:43433
I20260808 02:17:16.046347 19908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.253:43433 every 8 connection(s)
I20260808 02:17:16.046669 19803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
I20260808 02:17:16.047683 19909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:16.049284 19909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:17:16.055154 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 19803
I20260808 02:17:16.055248 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-1/wal/instance
I20260808 02:17:16.055438 19691 external_mini_cluster.cc:1428] Setting key d11a300b9df4738087684f96602070b1
I20260808 02:17:16.058391 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.050083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.19.58.254:40555)
0808 02:17:16.050146 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.050148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.050391 (+ 243us) server_negotiation.cc:407] Connection header received
0808 02:17:16.050861 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.050862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.050876 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.050927 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.051569 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.052120 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.053134 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.053384 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.053444 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.055610 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.055612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.055612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.055620 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.056887 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.057253 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.057254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.057255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.057307 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.057610 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.057611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.057612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.057773 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.057868 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.058200 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.058286 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:17:16.058544 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.050052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40555 (local address 127.0.0.1:52312)
0808 02:17:16.050230 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.050268 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:16.050771 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:16.050990 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:16.050993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:16.051224 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:16.051463 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.051468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.052216 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.052217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.053055 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.053058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.053477 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.053479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.053587 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.054050 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:16.054058 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:16.055526 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:16.056965 (+ 1439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.056972 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.056974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.057185 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.057412 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.057413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.057414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.057538 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.057872 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:16.057880 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:16.058060 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 02:17:16.058174 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 02:17:16.058326 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260808 02:17:16.059113 19909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.58.252@KRBTEST.COM; defaulting to no policy
I20260808 02:17:16.066500 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.059345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:43433 (local address 127.0.0.1:53992)
0808 02:17:16.059383 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.059393 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:16.059430 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:16.059671 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:16.059672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:16.059816 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:16.059867 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.059869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.061152 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.061153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.061634 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.061635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.062263 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.062264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.062319 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.062705 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:16.062709 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:16.063983 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:16.065613 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.065614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.065614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.065797 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.065954 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.065955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.065955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.066002 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.066204 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:16.066204 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:16.066275 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 02:17:16.066366 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 02:17:16.066438 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 02:17:16.066581 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.059425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53992 (local address 127.19.58.253:43433)
0808 02:17:16.059549 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.059554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.059569 (+ 15us) server_negotiation.cc:407] Connection header received
0808 02:17:16.059587 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.059588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.059600 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.059659 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.059951 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.061043 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.061704 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.062183 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.062257 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.064075 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.064084 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.064084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.064091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.065532 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.065860 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.065861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.065862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.065910 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.066042 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.066043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.066043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.066162 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.066216 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.066418 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.066483 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":100,"thread_start_us":78,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.252@KRBTEST.COM" created.
I20260808 02:17:16.067168 19909 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:16.067512 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.067396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60472)
0808 02:17:16.067443 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.067492 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 02:17:16.067646 19830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
I20260808 02:17:16.068118 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.068023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60484)
0808 02:17:16.068068 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.068099 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 02:17:16.068210 19909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:36155: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab.
Entry for principal kudu/127.19.58.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab.
I20260808 02:17:16.080621 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.252:36155
--webserver_interface=127.19.58.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab
--principal=kudu/127.19.58.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:34149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:40555,127.19.58.253:43433,127.19.58.252:36155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:17:16.117443 19909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } attempt: 1
I20260808 02:17:16.117997 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.117822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60496)
0808 02:17:16.117887 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.117956 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 02:17:16.118335 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.118248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60512)
0808 02:17:16.118287 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.118314 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 02:17:16.118441 19909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:36155: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
I20260808 02:17:16.123890 19793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } attempt: 3
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
I20260808 02:17:16.130255 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.124180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:43433 (local address 127.0.0.1:53996)
0808 02:17:16.124234 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.124243 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:16.124288 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:16.124416 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:16.124417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:16.124578 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:16.124641 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.124643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.125278 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.125280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.125802 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.125804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.126169 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.126170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.126246 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.126697 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:16.126701 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:16.127902 (+ 1201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:16.129296 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.129298 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.129298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.129455 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.129688 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.129689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.129689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.129738 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.130001 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:16.130001 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:16.130042 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:17:16.130092 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 02:17:16.130166 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 02:17:16.130345 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.124223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53996 (local address 127.19.58.253:43433)
0808 02:17:16.124288 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.124290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.124300 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:17:16.124346 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.124346 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.124358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.124406 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.124714 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.125207 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.125881 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.126105 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.126161 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.128005 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.128007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.128007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.128013 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.129225 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.129541 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.129543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.129543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.129617 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.129782 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.129783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.129784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.129925 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.129993 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.130128 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.130238 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 02:17:16.130733 19793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:16.131031 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.130934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60516)
0808 02:17:16.130980 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.131006 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 02:17:16.131610 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.131523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60518)
0808 02:17:16.131563 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.131588 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 02:17:16.131688 19793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:36155: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
W20260808 02:17:16.162437 19921 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:16.162653 19921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:16.162678 19921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:16.162703 19921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 02:17:16.162833 19793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } attempt: 1
I20260808 02:17:16.163259 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.163142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60522)
0808 02:17:16.163198 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.163229 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 02:17:16.163599 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.163523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60530)
0808 02:17:16.163559 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.163582 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 02:17:16.163722 19793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:36155: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
W20260808 02:17:16.163944 19921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:16.163980 19921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:16.163992 19921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:16.164004 19921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:16.164021 19921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:16.165474 19921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:34149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:40555,127.19.58.253:43433,127.19.58.252:36155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab
--principal=kudu/127.19.58.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.252:36155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.58.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:16.165660 19921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:16.165840 19921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:16.170687 19931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:16.170687 19933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:16.170861 19929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:16.171440 19921 server_base.cc:1034] running on GCE node
I20260808 02:17:16.171671 19921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:16.171985 19921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:16.173133 19921 hybrid_clock.cc:648] HybridClock initialized: now 1786155436173121 us; error 31 us; skew 500 ppm
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:16.175537 19921 init.cc:377] Logged in from keytab as kudu/127.19.58.252@KRBTEST.COM (short username kudu)
I20260808 02:17:16.175998 19921 webserver.cc:460] Webserver started at http://127.19.58.252:40503/ using document root <none> and password file <none>
I20260808 02:17:16.176193 19921 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:16.176237 19921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:16.176360 19921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:16.177350 19921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/data/instance:
uuid: "b5516b5678ab42fa89d450a632904029"
format_stamp: "Formatted at 2026-08-08 02:17:16 on dist-test-slave-1l3l"
server_key: "e65a130561b62996ebf565d7a7fcb6d9"
server_key_iv: "91118ed77c6c9298c422557f7d673a11"
server_key_version: "encryptionkey@0"
I20260808 02:17:16.177678 19921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/wal/instance:
uuid: "b5516b5678ab42fa89d450a632904029"
format_stamp: "Formatted at 2026-08-08 02:17:16 on dist-test-slave-1l3l"
server_key: "e65a130561b62996ebf565d7a7fcb6d9"
server_key_iv: "91118ed77c6c9298c422557f7d673a11"
server_key_version: "encryptionkey@0"
I20260808 02:17:16.178893 19921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 02:17:16.179507 19943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:16.179657 19921 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:17:16.179760 19921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/wal
uuid: "b5516b5678ab42fa89d450a632904029"
format_stamp: "Formatted at 2026-08-08 02:17:16 on dist-test-slave-1l3l"
server_key: "e65a130561b62996ebf565d7a7fcb6d9"
server_key_iv: "91118ed77c6c9298c422557f7d673a11"
server_key_version: "encryptionkey@0"
I20260808 02:17:16.179821 19921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:16.186594 19909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } attempt: 2
I20260808 02:17:16.187001 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.186879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60532)
0808 02:17:16.186930 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.186972 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 02:17:16.187323 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.187221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60542)
0808 02:17:16.187262 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.187290 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 02:17:16.187453 19909 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:36155: Network error: Client connection negotiation failed: client connection to 127.19.58.252:36155: connect: Connection refused (error 111)
I20260808 02:17:16.193336 19921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:16.194970 19921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:16.195071 19921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:16.198674 19921 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.252:36155
I20260808 02:17:16.198715 20020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.252:36155 every 8 connection(s)
I20260808 02:17:16.199018 19921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
I20260808 02:17:16.199976 20021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:16.201376 20021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:16.205451 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 19921
I20260808 02:17:16.205559 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786155170542577-19691-0/minicluster-data/master-2/wal/instance
I20260808 02:17:16.205709 19691 external_mini_cluster.cc:1428] Setting key cc70392f4b9c03bcc1df4ffd8dd69cf3
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:17:16.209892 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.201973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.19.58.254:40555)
0808 02:17:16.202044 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.202046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.202208 (+ 162us) server_negotiation.cc:407] Connection header received
0808 02:17:16.202645 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.202646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.202656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.202703 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.203248 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.203737 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.204652 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.204837 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.204891 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.207136 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.207137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.207138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.207144 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.208433 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.208940 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.208941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.208941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.209008 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.209283 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.209284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.209285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.209404 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.209453 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.209760 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.209827 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 02:17:16.210101 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.201942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:40555 (local address 127.0.0.1:52324)
0808 02:17:16.202095 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.202134 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:16.202572 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:16.202751 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:16.202754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:16.203004 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:16.203184 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.203188 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.203806 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.203807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.204576 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.204578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.204908 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.204908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.205023 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.205442 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:16.205449 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:16.207049 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:16.208549 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.208557 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.208558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.208860 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.209097 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.209098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.209099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.209204 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.209468 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:16.209476 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:16.209619 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 02:17:16.209727 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 02:17:16.209886 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260808 02:17:16.210700 20021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
I20260808 02:17:16.214185 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.206082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60548 (local address 127.19.58.252:36155)
0808 02:17:16.206246 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.206250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.206295 (+ 45us) server_negotiation.cc:407] Connection header received
0808 02:17:16.206332 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.206335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.206355 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.206469 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.207152 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.207857 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.208820 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.209183 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.209234 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.211689 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.211696 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.211697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.211722 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.213139 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.213442 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.213444 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.213445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.213504 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.213671 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.213672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.213673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.213835 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.213974 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.214017 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.214089 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260808 02:17:16.216737 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.210914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:43433 (local address 127.0.0.1:54012)
0808 02:17:16.210960 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.210972 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:16.211013 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:16.211118 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:16.211119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:16.211247 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:16.211299 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.211301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.211931 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.211932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.212414 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.212415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.212737 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.212738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.212802 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.213145 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:16.213149 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:16.214292 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:16.215803 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.215804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.215804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.215998 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.216158 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.216159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.216159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.216217 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.216466 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:16.216467 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:16.216508 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:17:16.216567 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:17:16.216649 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 02:17:16.216743 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.210987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.19.58.253:43433)
0808 02:17:16.211027 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.211029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.211037 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:17:16.211063 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.211063 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.211073 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.211114 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.211359 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.211867 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.212484 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.212682 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.212734 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.214367 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.214368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.214369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.214375 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.215738 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.216057 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.216057 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.216057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.216100 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.216276 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.216277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.216277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.216406 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.216462 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.216596 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.216659 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:17:16.217068 20021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
I20260808 02:17:16.221760 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.214918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52336 (local address 127.19.58.254:40555)
0808 02:17:16.214950 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.214952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.214959 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:17:16.214996 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.214997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.215005 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.215049 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.215490 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.216069 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.216674 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.216936 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.216992 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.219277 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.219278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.219278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.219284 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.220689 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.221038 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.221039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.221039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.221086 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.221267 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.221268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.221268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.221393 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.221528 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.221599 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.221672 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:17:16.223500 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.217282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60554)
0808 02:17:16.217324 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.217334 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:16.217378 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:16.217491 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:16.217492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:16.217598 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:16.217624 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.217625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.218446 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.218447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.219085 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.219087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.219671 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.219672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.219749 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.220036 (+ 287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:16.220040 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:16.221179 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:16.222678 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.222679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.222679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.222823 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.222996 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.222997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.222997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.223060 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.223253 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:16.223254 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:16.223294 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 02:17:16.223362 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:17:16.223445 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 02:17:16.223603 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.217316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.19.58.252:36155)
0808 02:17:16.217344 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.217346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.217365 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:17:16.217436 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.217436 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.217444 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.217486 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.217688 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.218376 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.219154 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.219408 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.219453 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.221255 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.221256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.221256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.221262 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.222616 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.222867 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.222868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.222868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.222927 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.223119 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.223120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.223120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.223203 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.223246 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.223386 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.223502 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":15}
I20260808 02:17:16.224740 20021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029: Bootstrap starting.
I20260808 02:17:16.225574 20021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:16.225828 20021 log.cc:824] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029: Log is configured to *not* fsync() on all Append() calls
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
I20260808 02:17:16.226576 20021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029: No bootstrap required, opened a new log
I20260808 02:17:16.228295 20021 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } }
I20260808 02:17:16.228456 20021 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5516b5678ab42fa89d450a632904029, State: Initialized, Role: FOLLOWER
I20260808 02:17:16.228545 20021 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } }
I20260808 02:17:16.228845 20021 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:16.228806 20030 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } } }
I20260808 02:17:16.228917 20030 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:16.229391 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.222558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.19.58.253:43433)
0808 02:17:16.222590 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.222592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.222599 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:17:16.222617 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.222618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.222626 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.222676 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.223384 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.224019 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.224612 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.224829 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.224884 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.226919 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.226920 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.226920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.226926 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.228172 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.228573 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.228573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.228574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.228623 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.228845 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.228846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.228847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.228970 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.229093 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.229221 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.229297 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 02:17:16.230679 20048 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 02:17:16.230702 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 02:17:16.230733 20048 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:16.231762 19753 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a8b22e76871440dbbba2decb4e903140"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52336
I20260808 02:17:16.230867 19921 master_runner.cc:186] Error getting master registration for 127.19.58.254:40555: OK, instance_id { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" instance_seqno: 1786155435898947 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:17:16.232762 19921 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:17:16.238874 19793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } attempt: 2
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155435, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
I20260808 02:17:16.246140 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.239127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60556)
0808 02:17:16.239175 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.239186 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:16.239232 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:16.239371 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:16.239372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:16.239536 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:16.239580 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.239581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.240489 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.240490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.241279 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.241281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.241705 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.241706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.241757 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.242138 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:16.242143 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:16.243595 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:16.245097 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.245097 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.245098 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.245310 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.245511 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.245512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.245512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.245573 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.245837 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:16.245838 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:16.245887 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 02:17:16.245954 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 02:17:16.246040 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 02:17:16.246140 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.239221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.19.58.252:36155)
0808 02:17:16.239266 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.239268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.239274 (+ 6us) server_negotiation.cc:407] Connection header received
0808 02:17:16.239293 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.239294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.239302 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.239353 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.239650 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.240423 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.241347 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.241644 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.241708 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.243678 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.243680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.243680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.243687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.245024 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.245394 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.245395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.245395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.245454 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.245627 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.245628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.245628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.245781 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.245841 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.245980 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.246050 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 02:17:16.247692 19793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140: Bootstrap starting.
I20260808 02:17:16.248353 19793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:16.248613 19793 log.cc:824] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:16.249287 19793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140: No bootstrap required, opened a new log
I20260808 02:17:16.250234 19793 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } }
I20260808 02:17:16.250340 19793 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8b22e76871440dbbba2decb4e903140, State: Initialized, Role: FOLLOWER
I20260808 02:17:16.250422 19793 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } }
I20260808 02:17:16.250664 20051 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } } }
I20260808 02:17:16.250710 19793 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:16.250751 20051 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:16.250852 19753 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 02:17:16.250970 19753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:16.251801 19753 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } }
W20260808 02:17:16.252737 20070 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:16.252790 20070 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:16.253206 19987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8b22e76871440dbbba2decb4e903140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b5516b5678ab42fa89d450a632904029"
I20260808 02:17:16.253316 19987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:16.254144 19987 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8b22e76871440dbbba2decb4e903140 in term 1.
I20260808 02:17:16.254436 19713 leader_election.cc:302] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8b22e76871440dbbba2decb4e903140, b5516b5678ab42fa89d450a632904029; no voters:
I20260808 02:17:16.254549 20051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:16.254664 20051 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [term 1 LEADER]: Becoming Leader. State: Replica: a8b22e76871440dbbba2decb4e903140, State: Running, Role: LEADER
I20260808 02:17:16.254735 20051 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } }
I20260808 02:17:16.255110 20051 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8b22e76871440dbbba2decb4e903140. Latest consensus state: current_term: 1 leader_uuid: "a8b22e76871440dbbba2decb4e903140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } } }
I20260808 02:17:16.255189 20051 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:16.255458 19753 leader_election.cc:288] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [CANDIDATE]: Term 1 election: Requested vote from peers 119ad6ac005d4a3d9e8efcd4cc862bcc (127.19.58.253:43433), b5516b5678ab42fa89d450a632904029 (127.19.58.252:36155)
I20260808 02:17:16.253418 19681 master_runner.cc:186] Error getting master registration for 127.19.58.253:43433: OK, instance_id { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" instance_seqno: 1786155436045251 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:17:16.255666 19681 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:17:16.257557 20076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:16.257767 20076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:16.260732 19987 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [term 1 FOLLOWER]: Refusing update from remote peer a8b22e76871440dbbba2decb4e903140: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:16.261113 20072 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:16.262444 20030 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8b22e76871440dbbba2decb4e903140. Latest consensus state: current_term: 1 leader_uuid: "a8b22e76871440dbbba2decb4e903140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } } }
I20260808 02:17:16.262537 20030 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:16.270126 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.257434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.19.58.252:36155)
0808 02:17:16.257467 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.257468 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.260680 (+ 3212us) server_negotiation.cc:407] Connection header received
0808 02:17:16.260762 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.260763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.260772 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.260799 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.261664 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.262314 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.263295 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.263552 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.263591 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.265632 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.265634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.265634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.265641 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.267389 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.268596 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.268597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.268597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.268670 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.269223 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.269224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.269225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.269365 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.269454 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.269958 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.270035 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:17:16.272042 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.256663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.19.58.253:43433)
0808 02:17:16.256704 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.256706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.257338 (+ 632us) server_negotiation.cc:407] Connection header received
0808 02:17:16.257476 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.257477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.257502 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.257533 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.258736 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.259476 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.263356 (+ 3880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.264325 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.264384 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.268723 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.268724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.268724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.268731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.270436 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.271011 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.271012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.271012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.271091 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.271363 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.271364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.271365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.271499 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.271563 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.271860 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.271947 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:17:16.274142 20030 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a8b22e76871440dbbba2decb4e903140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } } }
I20260808 02:17:16.274222 20030 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:16.274421 20086 mvcc.cc:205] Tried to move back new op lower bound from 7316092666920878080 to 7316092666900008960. Current Snapshot: MvccSnapshot[applied={T|T < 7316092666920878080}]
I20260808 02:17:16.274545 20051 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a8b22e76871440dbbba2decb4e903140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } } }
I20260808 02:17:16.274621 20051 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:16.274921 20084 mvcc.cc:205] Tried to move back new op lower bound from 7316092666920878080 to 7316092666900008960. Current Snapshot: MvccSnapshot[applied={T|T < 7316092666920878080}]
I20260808 02:17:16.278575 19909 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } attempt: 3
I20260808 02:17:16.280740 20076 catalog_manager.cc:1347] Generated new cluster ID: 7b6204743b22413ba923d5716694cd38
I20260808 02:17:16.280786 20076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 02:17:16 dist-test-slave-1l3l krb5kdc[19657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155436, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
I20260808 02:17:16.286854 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.278805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:36155 (local address 127.0.0.1:60574)
0808 02:17:16.278848 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:16.278858 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:16.278900 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:16.279023 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:16.279023 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:16.279169 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:16.279210 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.279212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.280126 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.280127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.280960 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:16.280963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.281553 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.281554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:16.281628 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.282116 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:16.282121 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:16.283698 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:16.285704 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.285705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.285705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.285929 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.286171 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:16.286172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:16.286173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:16.286243 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:16.286548 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:16.286549 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:16.286605 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 02:17:16.286661 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 02:17:16.286756 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 02:17:16.287523 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.278854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60574 (local address 127.19.58.252:36155)
0808 02:17:16.278916 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.278917 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.278936 (+ 19us) server_negotiation.cc:407] Connection header received
0808 02:17:16.278955 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.278956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.278965 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.279021 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.279277 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.280037 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.281118 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.281429 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.281485 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.283818 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.283820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.283820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.283826 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.285595 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.286022 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.286023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.286024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.286093 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.286315 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.286316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.286316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.286452 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.286506 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.287361 (+ 855us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.287435 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 02:17:16.289193 19909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc: Bootstrap starting.
I20260808 02:17:16.289914 19909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:16.290190 19909 log.cc:824] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:16.290367 20076 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:16.291028 20076 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:16.291190 19909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc: No bootstrap required, opened a new log
I20260808 02:17:16.293185 19909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } }
I20260808 02:17:16.293313 19909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119ad6ac005d4a3d9e8efcd4cc862bcc, State: Initialized, Role: FOLLOWER
I20260808 02:17:16.293381 19909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } }
I20260808 02:17:16.293679 19909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:16.295070 19863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:16.295213 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:16.257098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52348 (local address 127.19.58.254:40555)
0808 02:17:16.257146 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.257148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.260534 (+ 3386us) server_negotiation.cc:407] Connection header received
0808 02:17:16.260599 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.260600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.260609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.260659 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.261364 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.262048 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.262981 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:16.263263 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:16.263299 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:16.271048 (+ 7749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:16.271049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:16.271049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:16.271056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:16.272619 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.280458 (+ 7839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.280460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.280460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.280517 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:16.289761 (+ 9244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:16.289762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:16.289763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:16.289915 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:16.289998 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:16.295020 (+ 5022us) server_negotiation.cc:299] Negotiation successful
0808 02:17:16.295124 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 02:17:16.294925 19864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8b22e76871440dbbba2decb4e903140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc"
I20260808 02:17:16.295951 19863 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [term 1 FOLLOWER]: Refusing update from remote peer a8b22e76871440dbbba2decb4e903140: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:16.296512 20051 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:16.297325 19864 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8b22e76871440dbbba2decb4e903140 in term 1.
I20260808 02:17:16.300034 20116 mvcc.cc:205] Tried to move back new op lower bound from 7316092667037073408 to 7316092666900008960. Current Snapshot: MvccSnapshot[applied={T|T < 7316092666920878080}]
I20260808 02:17:16.300199 20121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8b22e76871440dbbba2decb4e903140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } } }
I20260808 02:17:16.300320 20121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:16.300410 20121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8b22e76871440dbbba2decb4e903140. Latest consensus state: current_term: 1 leader_uuid: "a8b22e76871440dbbba2decb4e903140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } } }
I20260808 02:17:16.300505 20121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:16.300575 20121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a8b22e76871440dbbba2decb4e903140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } } }
I20260808 02:17:16.300653 20121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:17:16.301133 20111 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:16.301225 20111 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:16.309294 20051 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a8b22e76871440dbbba2decb4e903140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8b22e76871440dbbba2decb4e903140" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 40555 } } peers { permanent_uuid: "119ad6ac005d4a3d9e8efcd4cc862bcc" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 43433 } } peers { permanent_uuid: "b5516b5678ab42fa89d450a632904029" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 36155 } } }
I20260808 02:17:16.309384 20051 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:16.310460 20076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8b22e76871440dbbba2decb4e903140: Generated new TSK 0
I20260808 02:17:16.310619 20076 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:17:16.334950 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.333244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.19.58.253:43433)
0808 02:17:16.333288 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.333290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.334263 (+ 973us) server_negotiation.cc:407] Connection header received
0808 02:17:16.334367 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.334369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.334380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.334414 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.334870 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54042: BlockingRecv error: recv got EOF from 127.0.0.1:54042 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 02:17:16.335772 19797 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.333659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52362 (local address 127.19.58.254:40555)
0808 02:17:16.333697 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.333699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.335191 (+ 1492us) server_negotiation.cc:407] Connection header received
0808 02:17:16.335286 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.335287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.335315 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.335345 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.335688 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52362: BlockingRecv error: recv got EOF from 127.0.0.1:52362 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 02:17:16.336488 20026 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.333998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.19.58.252:36155)
0808 02:17:16.334036 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.334038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.335973 (+ 1935us) server_negotiation.cc:407] Connection header received
0808 02:17:16.336097 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.336098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.336109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.336138 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.336432 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60580: BlockingRecv error: recv got EOF from 127.0.0.1:60580 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 02:17:16.338415 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.337633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54050 (local address 127.19.58.253:43433)
0808 02:17:16.337667 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.337668 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.337918 (+ 250us) server_negotiation.cc:407] Connection header received
0808 02:17:16.338025 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.338026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.338044 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.338074 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.338373 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54050: BlockingRecv error: recv got EOF from 127.0.0.1:54050 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 02:17:16.341509 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.340765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54062 (local address 127.19.58.253:43433)
0808 02:17:16.340796 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.340798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.341037 (+ 239us) server_negotiation.cc:407] Connection header received
0808 02:17:16.341131 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.341132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.341142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.341169 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.341471 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54062: BlockingRecv error: recv got EOF from 127.0.0.1:54062 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 02:17:16.345235 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.344472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54074 (local address 127.19.58.253:43433)
0808 02:17:16.344503 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.344505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.344744 (+ 239us) server_negotiation.cc:407] Connection header received
0808 02:17:16.344836 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.344837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.344847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.344873 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.345195 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54074: BlockingRecv error: recv got EOF from 127.0.0.1:54074 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 02:17:16.350943 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.350197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.19.58.253:43433)
0808 02:17:16.350229 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.350230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.350464 (+ 234us) server_negotiation.cc:407] Connection header received
0808 02:17:16.350556 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.350557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.350566 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.350593 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.350904 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54082: BlockingRecv error: recv got EOF from 127.0.0.1:54082 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 02:17:16.360793 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.360018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.19.58.253:43433)
0808 02:17:16.360053 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.360055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.360298 (+ 243us) server_negotiation.cc:407] Connection header received
0808 02:17:16.360389 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.360390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.360400 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.360428 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.360736 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54092: BlockingRecv error: recv got EOF from 127.0.0.1:54092 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 02:17:16.379056 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.378058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.19.58.253:43433)
0808 02:17:16.378099 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.378101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.378404 (+ 303us) server_negotiation.cc:407] Connection header received
0808 02:17:16.378515 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.378516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.378529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.378561 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.378996 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54102: BlockingRecv error: recv got EOF from 127.0.0.1:54102 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 02:17:16.412930 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.412250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54114 (local address 127.19.58.253:43433)
0808 02:17:16.412292 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.412294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.412428 (+ 134us) server_negotiation.cc:407] Connection header received
0808 02:17:16.412477 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.412478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.412490 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.412537 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.412875 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54114: BlockingRecv error: recv got EOF from 127.0.0.1:54114 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 02:17:16.478607 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.477885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54124 (local address 127.19.58.253:43433)
0808 02:17:16.477961 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.477963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.478057 (+ 94us) server_negotiation.cc:407] Connection header received
0808 02:17:16.478106 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.478106 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.478117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.478180 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.478537 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54124: BlockingRecv error: recv got EOF from 127.0.0.1:54124 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 02:17:16.608428 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.607744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54134 (local address 127.19.58.253:43433)
0808 02:17:16.607806 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.607809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.607882 (+ 73us) server_negotiation.cc:407] Connection header received
0808 02:17:16.607974 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.607975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.607988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.608037 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.608372 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54134: BlockingRecv error: recv got EOF from 127.0.0.1:54134 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 02:17:16.866250 19916 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:16.865637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.19.58.253:43433)
0808 02:17:16.865703 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:16.865706 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:16.865719 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:17:16.865781 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:16.865782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:16.865793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:16.865854 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:16.866182 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54150: BlockingRecv error: recv got EOF from 127.0.0.1:54150 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 02:17:17.231141 20048 catalog_manager.cc:1259] Loaded cluster ID: 7b6204743b22413ba923d5716694cd38
I20260808 02:17:17.231216 20048 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029: loading cluster ID for follower catalog manager: success
I20260808 02:17:17.232054 20048 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029: acquiring CA information for follower catalog manager: success
I20260808 02:17:17.232337 20048 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b5516b5678ab42fa89d450a632904029: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:17:17.301710 20111 catalog_manager.cc:1259] Loaded cluster ID: 7b6204743b22413ba923d5716694cd38
I20260808 02:17:17.301787 20111 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc: loading cluster ID for follower catalog manager: success
I20260808 02:17:17.302546 20111 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc: acquiring CA information for follower catalog manager: success
I20260808 02:17:17.302793 20111 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 119ad6ac005d4a3d9e8efcd4cc862bcc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:17:17.381644 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:17.379327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54166 (local address 127.19.58.253:43433)
0808 02:17:17.379468 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:17.379469 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:17.379480 (+ 11us) server_negotiation.cc:407] Connection header received
0808 02:17:17.379549 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:17.379549 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:17.379598 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:17.379658 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:17.379825 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:17.380316 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:17.381021 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:17.381210 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:17.381253 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:17.381353 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:17.381475 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:17.381529 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:17.381543 (+ 14us) server_negotiation.cc:299] Negotiation successful
0808 02:17:17.381559 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260808 02:17:17.385448 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:17.383282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60582 (local address 127.19.58.252:36155)
0808 02:17:17.383373 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:17.383375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:17.383385 (+ 10us) server_negotiation.cc:407] Connection header received
0808 02:17:17.383475 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:17.383476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:17.383511 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:17.383564 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:17.383640 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:17.384142 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:17.384809 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:17.385028 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:17.385075 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:17.385181 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:17.385280 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:17.385332 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:17.385362 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 02:17:17.385378 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":48,"threads_started":1}
I20260808 02:17:17.389688 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:17.387250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60598 (local address 127.19.58.252:36155)
0808 02:17:17.387284 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:17.387285 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:17.387293 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:17:17.387313 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:17.387314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:17.387323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:17.387371 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:17.387480 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:17.388029 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:17.389001 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:17.389266 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:17.389333 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:17.389454 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:17.389482 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:17.389526 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:17.389605 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 02:17:17.389624 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:17:17.389767 20219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:17.386995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54172 (local address 127.19.58.253:43433)
0808 02:17:17.387037 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:17.387040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:17.387131 (+ 91us) server_negotiation.cc:407] Connection header received
0808 02:17:17.387229 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:17.387230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:17.387240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:17.387267 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:17.387444 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:17.388191 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:17.389111 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:17.389314 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:17.389368 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:17.389517 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:17.389549 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:17.389589 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:17.389681 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 02:17:17.389703 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:17:17.390221 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:17.387248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.19.58.254:40555)
0808 02:17:17.387411 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:17.387413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:17.387420 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:17:17.387465 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:17.387466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:17.387513 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:17.387591 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:17.387698 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:17.388411 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:17.389191 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:17.389485 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:17.389550 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:17.389717 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:17.389984 (+ 267us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:17.390081 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:17.390123 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 02:17:17.390145 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":62,"threads_started":1}
I20260808 02:17:17.391552 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 19681
I20260808 02:17:17.403955 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 19803
I20260808 02:17:17.414008 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 19921
2026-08-08T02:17:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2862 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 02:17:17 dist-test-slave-1l3l krb5kdc[20242](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 02:17:17 dist-test-slave-1l3l krb5kdc[20242](info): set up 2 sockets
Aug 08 02:17:17 dist-test-slave-1l3l krb5kdc[20242](info): commencing operation
krb5kdc: starting...
W20260808 02:17:18.530526 19691 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/test-user.keytab.
Aug 08 02:17:18 dist-test-slave-1l3l krb5kdc[20242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T02:17:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T02:17:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.58.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
Entry for principal kudu/127.19.58.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab.
I20260808 02:17:18.642421 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:41719
--webserver_interface=127.19.58.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:41719,127.19.58.253:41665,127.19.58.252:34919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 02:17:18.722043 20265 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:18.722244 20265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:18.722267 20265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:18.722292 20265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:18.723380 20265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:18.723419 20265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:18.723438 20265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:18.723452 20265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:18.723469 20265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:18.725004 20265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:41719,127.19.58.253:41665,127.19.58.252:34919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.254.keytab
--principal=kudu/127.19.58.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.254:41719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.58.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:18.725243 20265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:18.725446 20265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:18.729930 20277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:18.729972 20274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:18.730011 20285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:18.730288 20265 server_base.cc:1034] running on GCE node
I20260808 02:17:18.730530 20265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:18.730755 20265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:18.731887 20265 hybrid_clock.cc:648] HybridClock initialized: now 1786155438731870 us; error 33 us; skew 500 ppm
Aug 08 02:17:18 dist-test-slave-1l3l krb5kdc[20242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:18.734253 20265 init.cc:377] Logged in from keytab as kudu/127.19.58.254@KRBTEST.COM (short username kudu)
I20260808 02:17:18.734714 20265 webserver.cc:460] Webserver started at http://127.19.58.254:36295/ using document root <none> and password file <none>
I20260808 02:17:18.734937 20265 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:18.734983 20265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:18.735121 20265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:18.736248 20265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/data/instance:
uuid: "bda02d487386484d850c91787245f732"
format_stamp: "Formatted at 2026-08-08 02:17:18 on dist-test-slave-1l3l"
server_key: "7237588b657f6bb5b2bf69d6ca4b688a"
server_key_iv: "acc07143e139e6cc4496afca5b849041"
server_key_version: "encryptionkey@0"
I20260808 02:17:18.736584 20265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/wal/instance:
uuid: "bda02d487386484d850c91787245f732"
format_stamp: "Formatted at 2026-08-08 02:17:18 on dist-test-slave-1l3l"
server_key: "7237588b657f6bb5b2bf69d6ca4b688a"
server_key_iv: "acc07143e139e6cc4496afca5b849041"
server_key_version: "encryptionkey@0"
I20260808 02:17:18.737836 20265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:17:18.738519 20296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:18.738684 20265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 02:17:18.738765 20265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/wal
uuid: "bda02d487386484d850c91787245f732"
format_stamp: "Formatted at 2026-08-08 02:17:18 on dist-test-slave-1l3l"
server_key: "7237588b657f6bb5b2bf69d6ca4b688a"
server_key_iv: "acc07143e139e6cc4496afca5b849041"
server_key_version: "encryptionkey@0"
I20260808 02:17:18.738818 20265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:18.753145 20265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:18.755180 20265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:18.755286 20265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:18.759032 20265 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.254:41719
I20260808 02:17:18.759075 20375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.254:41719 every 8 connection(s)
I20260808 02:17:18.759375 20265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/data/info.pb
I20260808 02:17:18.760317 20376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:18.761838 20376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:18.766819 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 20265
I20260808 02:17:18.766916 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-0/wal/instance
I20260808 02:17:18.767114 19691 external_mini_cluster.cc:1428] Setting key 581d72a14f55419f989543fce06142a0
Aug 08 02:17:18 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:17:18.770946 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.762425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:41719 (local address 127.0.0.1:48916)
0808 02:17:18.762575 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.762606 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:18.763070 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:18.763333 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:18.763335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:18.763583 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:18.763769 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:18.763774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.764768 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.764769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:18.765420 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:18.765422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.765886 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.765887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:18.765956 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:18.766347 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:18.766355 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:18.767871 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:18.769594 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:18.769601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:18.769602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:18.769787 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:18.770004 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:18.770004 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:18.770005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:18.770091 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:18.770395 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:18.770400 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:18.770464 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 02:17:18.770549 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:17:18.770703 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260808 02:17:18.770946 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.762580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.19.58.254:41719)
0808 02:17:18.762680 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:18.762684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:18.762715 (+ 31us) server_negotiation.cc:407] Connection header received
0808 02:17:18.763221 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:18.763224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:18.763240 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:18.763314 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:18.763858 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.764684 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.765499 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.765798 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.765907 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:18.767951 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:18.767959 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:18.767960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:18.767966 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:18.769500 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:18.769858 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:18.769862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:18.769862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:18.769945 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:18.770145 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:18.770146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:18.770146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:18.770334 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:18.770431 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:18.770593 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 02:17:18.770766 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":76,"thread_start_us":39,"threads_started":1}
I20260808 02:17:18.771663 20376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:18.772006 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.771906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:41665 (local address 127.0.0.1:33030)
0808 02:17:18.771945 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.771985 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 02:17:18.772130 20305 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.58.253:41665: connect: Connection refused (error 111)
I20260808 02:17:18.772511 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.772426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:41665 (local address 127.0.0.1:33044)
0808 02:17:18.772464 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.772488 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 02:17:18.772605 20376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.253:41665: Network error: Client connection negotiation failed: client connection to 127.19.58.253:41665: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.19.58.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab.
Entry for principal kudu/127.19.58.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab.
I20260808 02:17:18.790251 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.253:41665
--webserver_interface=127.19.58.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab
--principal=kudu/127.19.58.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:41719,127.19.58.253:41665,127.19.58.252:34919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:17:18.821813 20376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } attempt: 1
I20260808 02:17:18.822311 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.822175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:41665 (local address 127.0.0.1:33056)
0808 02:17:18.822238 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.822279 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 02:17:18.822669 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.822582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:41665 (local address 127.0.0.1:33072)
0808 02:17:18.822623 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.822647 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 02:17:18.822757 20376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.253:41665: Network error: Client connection negotiation failed: client connection to 127.19.58.253:41665: connect: Connection refused (error 111)
W20260808 02:17:18.869505 20384 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:18.869697 20384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:18.869724 20384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:18.869746 20384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:18.870813 20384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:18.870851 20384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:18.870870 20384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:18.870886 20384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:18.870900 20384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:18.872340 20384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:41719,127.19.58.253:41665,127.19.58.252:34919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.253.keytab
--principal=kudu/127.19.58.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.253:41665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.58.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:18.872584 20384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:18.872776 20384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:18.877218 20401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:18.877393 20384 server_base.cc:1034] running on GCE node
W20260808 02:17:18.877274 20394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:18.877264 20392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:18.877719 20384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:18.877981 20384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:18.879119 20384 hybrid_clock.cc:648] HybridClock initialized: now 1786155438879106 us; error 26 us; skew 500 ppm
Aug 08 02:17:18 dist-test-slave-1l3l krb5kdc[20242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:18.881475 20384 init.cc:377] Logged in from keytab as kudu/127.19.58.253@KRBTEST.COM (short username kudu)
I20260808 02:17:18.881899 20384 webserver.cc:460] Webserver started at http://127.19.58.253:44623/ using document root <none> and password file <none>
I20260808 02:17:18.882107 20384 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:18.882155 20384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:18.882277 20384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:18.883256 20384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/data/instance:
uuid: "29a113b6c091496dbe82e12880e5dbc3"
format_stamp: "Formatted at 2026-08-08 02:17:18 on dist-test-slave-1l3l"
server_key: "0d4acb8b980977cdf7e87a3881d69270"
server_key_iv: "e0da1f142215b4a7a3e70981dc395952"
server_key_version: "encryptionkey@0"
I20260808 02:17:18.883581 20384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/wal/instance:
uuid: "29a113b6c091496dbe82e12880e5dbc3"
format_stamp: "Formatted at 2026-08-08 02:17:18 on dist-test-slave-1l3l"
server_key: "0d4acb8b980977cdf7e87a3881d69270"
server_key_iv: "e0da1f142215b4a7a3e70981dc395952"
server_key_version: "encryptionkey@0"
I20260808 02:17:18.884828 20384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 02:17:18.885480 20412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:18.885625 20384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 02:17:18.885704 20384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/wal
uuid: "29a113b6c091496dbe82e12880e5dbc3"
format_stamp: "Formatted at 2026-08-08 02:17:18 on dist-test-slave-1l3l"
server_key: "0d4acb8b980977cdf7e87a3881d69270"
server_key_iv: "e0da1f142215b4a7a3e70981dc395952"
server_key_version: "encryptionkey@0"
I20260808 02:17:18.885761 20384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:18.890889 20376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } attempt: 2
I20260808 02:17:18.891248 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.891120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:41665 (local address 127.0.0.1:33086)
0808 02:17:18.891175 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.891213 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 02:17:18.891574 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.891485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:41665 (local address 127.0.0.1:33100)
0808 02:17:18.891525 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.891552 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.253:41665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 02:17:18.891660 20376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.253:41665: Network error: Client connection negotiation failed: client connection to 127.19.58.253:41665: connect: Connection refused (error 111)
I20260808 02:17:18.894119 20384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:18.895742 20384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:18.895856 20384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:18.899514 20384 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.253:41665
I20260808 02:17:18.899561 20488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.253:41665 every 8 connection(s)
I20260808 02:17:18.899869 20384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/data/info.pb
I20260808 02:17:18.900753 20490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:18.902163 20490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:18.904692 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 20384
I20260808 02:17:18.904778 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-1/wal/instance
I20260808 02:17:18.904940 19691 external_mini_cluster.cc:1428] Setting key 2760e1a1b2235de7ddc25012abfcb85a
Aug 08 02:17:18 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:17:18.910255 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.902765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.19.58.254:41719)
0808 02:17:18.902822 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:18.902823 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:18.902997 (+ 174us) server_negotiation.cc:407] Connection header received
0808 02:17:18.903427 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:18.903428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:18.903439 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:18.903485 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:18.904060 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.904546 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.905527 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.905797 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.905867 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:18.907787 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:18.907788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:18.907788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:18.907795 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:18.908961 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:18.909411 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:18.909412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:18.909413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:18.909463 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:18.909705 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:18.909706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:18.909706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:18.909796 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:18.909839 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:18.910077 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 02:17:18.910166 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 02:17:18.910374 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.902732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:41719 (local address 127.0.0.1:48932)
0808 02:17:18.902885 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.902923 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:18.903345 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:18.903539 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:18.903542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:18.903800 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:18.903983 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:18.903988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.904620 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.904620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:18.905438 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:18.905441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.905878 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.905879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:18.905973 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:18.906395 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:18.906402 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:18.907687 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:18.909040 (+ 1353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:18.909048 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:18.909050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:18.909333 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:18.909516 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:18.909517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:18.909517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:18.909649 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:18.909831 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:18.909835 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:18.909949 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 02:17:18.910034 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:17:18.910178 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260808 02:17:18.910856 20490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.58.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.58.252@KRBTEST.COM" created.
Aug 08 02:17:18 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
I20260808 02:17:18.919438 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.911083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:41665 (local address 127.0.0.1:33104)
0808 02:17:18.911126 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.911135 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:18.911168 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:18.911384 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:18.911385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:18.911520 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:18.911580 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:18.911582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.912384 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.912385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:18.913140 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:18.913143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.913655 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.913657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:18.913733 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:18.914044 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:18.914048 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:18.916768 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:18.918481 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:18.918482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:18.918482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:18.918631 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:18.918826 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:18.918826 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:18.918827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:18.918902 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:18.919133 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:18.919134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:18.919175 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 02:17:18.919244 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 02:17:18.919336 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 02:17:18.919469 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.911126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33104 (local address 127.19.58.253:41665)
0808 02:17:18.911259 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:18.911264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:18.911277 (+ 13us) server_negotiation.cc:407] Connection header received
0808 02:17:18.911294 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:18.911295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:18.911305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:18.911368 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:18.911644 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.912300 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.913216 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.913589 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.913652 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:18.916838 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:18.916847 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:18.916848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:18.916854 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:18.918406 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:18.918705 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:18.918708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:18.918708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:18.918766 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:18.918960 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:18.918961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:18.918961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:18.919080 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:18.919136 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:18.919268 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 02:17:18.919383 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260808 02:17:18.920102 20490 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:18.920404 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.920279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50496)
0808 02:17:18.920328 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.920379 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 02:17:18.920536 20416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
I20260808 02:17:18.920925 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.920829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50500)
0808 02:17:18.920868 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.920903 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 02:17:18.921023 20490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:34919: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.58.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab.
Entry for principal kudu/127.19.58.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab.
I20260808 02:17:18.930320 19691 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDIPgti/build/release/bin/kudu
/tmp/dist-test-taskDIPgti/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.252:34919
--webserver_interface=127.19.58.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab
--principal=kudu/127.19.58.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.58.212:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.58.254:41719,127.19.58.253:41665,127.19.58.252:34919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 02:17:18.970250 20490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } attempt: 1
I20260808 02:17:18.970758 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.970614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50512)
0808 02:17:18.970674 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.970723 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 02:17:18.971084 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.971001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50522)
0808 02:17:18.971039 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.971064 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 02:17:18.971190 20490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:34919: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
I20260808 02:17:18.982833 20376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } attempt: 3
Aug 08 02:17:18 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
I20260808 02:17:18.989529 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.983246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33108 (local address 127.19.58.253:41665)
0808 02:17:18.983303 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:18.983305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:18.983317 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:17:18.983385 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:18.983387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:18.983399 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:18.983449 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:18.983777 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.984288 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.984971 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.985193 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.985249 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:18.987080 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:18.987082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:18.987082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:18.987089 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:18.988465 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:18.988823 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:18.988824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:18.988824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:18.988870 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:18.989055 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:18.989056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:18.989056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:18.989165 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:18.989215 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:18.989389 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 02:17:18.989463 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 02:17:18.989581 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.983185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:41665 (local address 127.0.0.1:33108)
0808 02:17:18.983255 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.983266 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:18.983323 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:18.983454 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:18.983455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:18.983614 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:18.983683 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:18.983685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.984362 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.984363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:18.984898 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:18.984900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:18.985254 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:18.985255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:18.985326 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:18.985739 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:18.985744 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:18.986991 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:18.988548 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:18.988549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:18.988549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:18.988763 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:18.988921 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:18.988922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:18.988922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:18.989002 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:18.989233 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:18.989234 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:18.989284 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 02:17:18.989369 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 02:17:18.989482 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 02:17:18.990013 20376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } has no permanent_uuid. Determining permanent_uuid...
I20260808 02:17:18.990326 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.990225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50524)
0808 02:17:18.990265 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.990298 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 02:17:18.990669 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:18.990581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50526)
0808 02:17:18.990622 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:18.990649 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 02:17:18.990751 20376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:34919: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
W20260808 02:17:19.010712 20498 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 02:17:19.010902 20498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 02:17:19.010919 20498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 02:17:19.010937 20498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 02:17:19.012003 20498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 02:17:19.012035 20498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 02:17:19.012048 20498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 02:17:19.012059 20498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 02:17:19.012073 20498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 02:17:19.013455 20498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.58.212:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.58.254:41719,127.19.58.253:41665,127.19.58.252:34919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/krb5kdc/kudu_127.19.58.252.keytab
--principal=kudu/127.19.58.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.58.252:34919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.58.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 01:00:39 UTC on ccycloud.quasar-tvudcf.root.comops.site
build id 4573648
I20260808 02:17:19.013648 20498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 02:17:19.013863 20498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 02:17:19.018265 20511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:19.018244 20508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 02:17:19.018239 20515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 02:17:19.018359 20498 server_base.cc:1034] running on GCE node
I20260808 02:17:19.018718 20498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 02:17:19.018978 20498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 02:17:19.020110 20498 hybrid_clock.cc:648] HybridClock initialized: now 1786155439020093 us; error 30 us; skew 500 ppm
I20260808 02:17:19.021872 20376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } attempt: 1
Aug 08 02:17:19 dist-test-slave-1l3l krb5kdc[20242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155439, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 02:17:19.022257 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.022144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50542)
0808 02:17:19.022193 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:19.022228 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 02:17:19.022555 20498 init.cc:377] Logged in from keytab as kudu/127.19.58.252@KRBTEST.COM (short username kudu)
I20260808 02:17:19.022603 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.022487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50554)
0808 02:17:19.022539 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:19.022573 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 02:17:19.022688 20376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:34919: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
I20260808 02:17:19.022984 20498 webserver.cc:460] Webserver started at http://127.19.58.252:33931/ using document root <none> and password file <none>
I20260808 02:17:19.023151 20498 fs_manager.cc:359] Metadata directory not provided
I20260808 02:17:19.023188 20498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 02:17:19.023358 20498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 02:17:19.024691 20498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/data/instance:
uuid: "d896c1df18424ac6bb91f847f3d29cb3"
format_stamp: "Formatted at 2026-08-08 02:17:19 on dist-test-slave-1l3l"
server_key: "6c4c6487de0067f0c049750e83cd05f6"
server_key_iv: "753bf025ce267a0975f5f15ea63b2a88"
server_key_version: "encryptionkey@0"
I20260808 02:17:19.025092 20498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/wal/instance:
uuid: "d896c1df18424ac6bb91f847f3d29cb3"
format_stamp: "Formatted at 2026-08-08 02:17:19 on dist-test-slave-1l3l"
server_key: "6c4c6487de0067f0c049750e83cd05f6"
server_key_iv: "753bf025ce267a0975f5f15ea63b2a88"
server_key_version: "encryptionkey@0"
I20260808 02:17:19.026396 20498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 02:17:19.027140 20524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 02:17:19.027314 20498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 02:17:19.027395 20498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/data,/tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/wal
uuid: "d896c1df18424ac6bb91f847f3d29cb3"
format_stamp: "Formatted at 2026-08-08 02:17:19 on dist-test-slave-1l3l"
server_key: "6c4c6487de0067f0c049750e83cd05f6"
server_key_iv: "753bf025ce267a0975f5f15ea63b2a88"
server_key_version: "encryptionkey@0"
I20260808 02:17:19.027468 20498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 02:17:19.039342 20490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } attempt: 2
I20260808 02:17:19.039728 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.039590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50566)
0808 02:17:19.039645 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:19.039679 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 02:17:19.040000 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.039923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50568)
0808 02:17:19.039958 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:19.039983 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 02:17:19.040088 20490 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.58.252:34919: Network error: Client connection negotiation failed: client connection to 127.19.58.252:34919: connect: Connection refused (error 111)
I20260808 02:17:19.050458 20498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 02:17:19.052134 20498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 02:17:19.052261 20498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 02:17:19.055989 20498 rpc_server.cc:307] RPC server started. Bound to: 127.19.58.252:34919
I20260808 02:17:19.056026 20596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.58.252:34919 every 8 connection(s)
I20260808 02:17:19.056332 20498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/data/info.pb
I20260808 02:17:19.057276 20598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 02:17:19.058713 20598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:17:19 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155439, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:17:19.065033 19691 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDIPgti/build/release/bin/kudu as pid 20498
I20260808 02:17:19.065138 19691 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDIPgti/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786155170542577-19691-0/minicluster-data/master-2/wal/instance
I20260808 02:17:19.065315 19691 external_mini_cluster.cc:1428] Setting key 46664eadf42a4ddaea635f24a9e72fdc
I20260808 02:17:19.067268 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.059336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.19.58.254:41719)
0808 02:17:19.059396 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.059398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.059592 (+ 194us) server_negotiation.cc:407] Connection header received
0808 02:17:19.060105 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.060106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.060119 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.060167 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.060778 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.061256 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.062202 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.062403 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.062461 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.064512 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.064514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.064514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.064522 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.065992 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.066362 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.066363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.066363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.066421 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.066655 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.066656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.066656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.066773 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.066830 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.067093 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.067186 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 02:17:19.067586 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.059284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.254:41719 (local address 127.0.0.1:48948)
0808 02:17:19.059465 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:19.059498 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:19.059994 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:19.060248 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:19.060251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:19.060488 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:19.060687 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:19.060692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.061338 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.061339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:19.062116 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:19.062119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.062486 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.062487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:19.062612 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.063062 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:19.063071 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:19.064421 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:19.066066 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:19.066074 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:19.066076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:19.066293 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:19.066481 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:19.066481 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:19.066482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:19.066594 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:19.066831 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:19.066839 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:19.066967 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 02:17:19.067055 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 02:17:19.067363 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":128,"thread_start_us":96,"threads_started":1}
I20260808 02:17:19.068179 20598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:17:19 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
Aug 08 02:17:19 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155439, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
I20260808 02:17:19.074084 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.065687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50584 (local address 127.19.58.252:34919)
0808 02:17:19.065812 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.065818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.065945 (+ 127us) server_negotiation.cc:407] Connection header received
0808 02:17:19.065996 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.065999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.066012 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.066075 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.066673 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.067355 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.068358 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.068771 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.068823 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.071314 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.071320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.071321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.071327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.072969 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.073272 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.073275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.073275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.073329 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.073558 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.073559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.073560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.073690 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.073809 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.073904 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.073973 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260808 02:17:19.074620 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.068532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.19.58.253:41665)
0808 02:17:19.068603 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.068605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.068614 (+ 9us) server_negotiation.cc:407] Connection header received
0808 02:17:19.068639 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.068640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.068648 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.068703 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.068935 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.069358 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.070273 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.070481 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.070524 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.072287 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.072288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.072289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.072297 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.073521 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.073834 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.073835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.073835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.073896 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.074144 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.074145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.074146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.074261 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.074299 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.074494 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.074555 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 02:17:19.074621 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.068468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.253:41665 (local address 127.0.0.1:33114)
0808 02:17:19.068516 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:19.068528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:19.068561 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:19.068703 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:19.068704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:19.068809 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:19.068862 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:19.068863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.069436 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.069437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:19.070202 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:19.070205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.070548 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.070549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:19.070624 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.070984 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:19.070989 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:19.072206 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:19.073601 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:19.073602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:19.073602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:19.073752 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:19.073973 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:19.073974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:19.073974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:19.074070 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:19.074349 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:19.074350 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:19.074390 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 02:17:19.074457 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 02:17:19.074538 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 02:17:19.075327 20598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 02:17:19 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155439, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.252@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
Aug 08 02:17:19 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.254@KRBTEST.COM
I20260808 02:17:19.081286 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.075542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50590)
0808 02:17:19.075627 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:19.075635 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:19.075682 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:19.075823 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:19.075824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:19.075958 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:19.075987 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:19.075989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.076649 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.076649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:19.077137 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:19.077139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.077496 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.077497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:19.077554 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.077888 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:19.077892 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:19.078972 (+ 1080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:19.080440 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:19.080441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:19.080441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:19.080599 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:19.080758 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:19.080759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:19.080759 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:19.080810 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:19.081025 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:19.081026 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:19.081066 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 02:17:19.081118 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 02:17:19.081178 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 02:17:19.081353 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.075592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.19.58.252:34919)
0808 02:17:19.075626 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.075628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.075671 (+ 43us) server_negotiation.cc:407] Connection header received
0808 02:17:19.075772 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.075772 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.075781 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.075819 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.076044 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.076578 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.077205 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.077422 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.077481 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.079051 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.079052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.079053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.079059 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.080370 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.080657 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.080658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.080658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.080708 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.080858 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.080859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.080859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.080952 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.081010 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.081156 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.081223 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:17:19.082527 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.075047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48952 (local address 127.19.58.254:41719)
0808 02:17:19.075082 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.075084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.075091 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:17:19.075112 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.075112 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.075120 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.075164 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.075676 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.076298 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.077031 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.077289 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.077337 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.079611 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.079613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.079613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.079619 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.080927 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.081257 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.081258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.081258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.081315 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.081594 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.081595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.081596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.081713 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.082323 (+ 610us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.082365 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.082439 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:17:19.083029 20598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3: Bootstrap starting.
I20260808 02:17:19.083926 20598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:19.084156 20598 log.cc:824] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:19.084923 20598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3: No bootstrap required, opened a new log
I20260808 02:17:19.086829 20598 raft_consensus.cc:348] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } }
I20260808 02:17:19.086948 20598 raft_consensus.cc:729] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d896c1df18424ac6bb91f847f3d29cb3, State: Initialized, Role: FOLLOWER
I20260808 02:17:19.087004 20598 consensus_queue.cc:260] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } }
Aug 08 02:17:19 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.58.253@KRBTEST.COM
I20260808 02:17:19.087311 20598 sys_catalog.cc:564] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:19.087263 20606 sys_catalog.cc:455] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } } }
I20260808 02:17:19.087371 20606 sys_catalog.cc:458] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 02:17:19.089553 20623 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:19.089653 20623 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:19.090428 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.083134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33124 (local address 127.19.58.253:41665)
0808 02:17:19.083174 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.083176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.083184 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:17:19.083243 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.083244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.083253 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.083296 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.083687 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.084199 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.084993 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.085212 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.085264 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.087506 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.087507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.087508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.087514 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.089115 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.089543 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.089544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.089544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.089605 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.089799 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.089800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.089800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.089931 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.090065 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.090266 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.090344 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 02:17:19.089733 20498 master_runner.cc:186] Error getting master registration for 127.19.58.254:41719: OK, instance_id { permanent_uuid: "bda02d487386484d850c91787245f732" instance_seqno: 1786155438758004 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:17:19.091996 20498 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:17:19.092170 19691 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 02:17:19.092929 20336 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bda02d487386484d850c91787245f732"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48952
I20260808 02:17:19.097836 20376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } attempt: 2
Aug 08 02:17:19 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.254@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
I20260808 02:17:19.104658 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.098127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50604 (local address 127.19.58.252:34919)
0808 02:17:19.098201 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.098202 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.098210 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:17:19.098227 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.098228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.098236 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.098279 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.098553 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.099290 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.099977 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.100166 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.100215 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.102001 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.102002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.102003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.102010 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.103490 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.103914 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.103915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.103915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.103969 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.104142 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.104143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.104143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.104294 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.104352 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.104493 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.104568 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 02:17:19.104672 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.098072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50604)
0808 02:17:19.098117 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:19.098127 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:19.098165 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:19.098288 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:19.098289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:19.098457 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:19.098498 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:19.098500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.099367 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.099368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:19.099906 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:19.099907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.100218 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.100219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:19.100272 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.100617 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:19.100622 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:19.101919 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:19.103568 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:19.103569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:19.103570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:19.103849 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:19.104027 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:19.104028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:19.104028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:19.104091 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:19.104354 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:19.104355 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:19.104400 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 02:17:19.104468 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 02:17:19.104564 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 02:17:19.106161 20376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732: Bootstrap starting.
I20260808 02:17:19.106752 20376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:19.107019 20376 log.cc:824] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:19.107697 20376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732: No bootstrap required, opened a new log
I20260808 02:17:19.108656 20376 raft_consensus.cc:348] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } }
I20260808 02:17:19.108762 20376 raft_consensus.cc:729] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bda02d487386484d850c91787245f732, State: Initialized, Role: FOLLOWER
I20260808 02:17:19.108834 20376 consensus_queue.cc:260] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } }
I20260808 02:17:19.109081 20626 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } } }
I20260808 02:17:19.109172 20626 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:19.109187 20336 raft_consensus.cc:482] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 02:17:19.109228 20336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:19.109781 20336 raft_consensus.cc:504] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } }
I20260808 02:17:19.109985 20336 leader_election.cc:288] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [CANDIDATE]: Term 1 election: Requested vote from peers 29a113b6c091496dbe82e12880e5dbc3 (127.19.58.253:41665), d896c1df18424ac6bb91f847f3d29cb3 (127.19.58.252:34919)
I20260808 02:17:19.109122 20376 sys_catalog.cc:564] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:19.110634 20560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bda02d487386484d850c91787245f732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d896c1df18424ac6bb91f847f3d29cb3"
I20260808 02:17:19.110744 20560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:19.111604 20560 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda02d487386484d850c91787245f732 in term 1.
W20260808 02:17:19.111817 20643 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:19.111871 20643 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:19.111994 20305 leader_election.cc:302] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bda02d487386484d850c91787245f732, d896c1df18424ac6bb91f847f3d29cb3; no voters:
I20260808 02:17:19.112115 20626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 02:17:19.112231 20626 raft_consensus.cc:686] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [term 1 LEADER]: Becoming Leader. State: Replica: bda02d487386484d850c91787245f732, State: Running, Role: LEADER
I20260808 02:17:19.112303 20626 consensus_queue.cc:237] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } }
I20260808 02:17:19.112677 20626 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bda02d487386484d850c91787245f732. Latest consensus state: current_term: 1 leader_uuid: "bda02d487386484d850c91787245f732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } } }
I20260808 02:17:19.112752 20626 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:19.112329 20265 master_runner.cc:186] Error getting master registration for 127.19.58.253:41665: OK, instance_id { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" instance_seqno: 1786155438898516 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 02:17:19.114797 20265 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 02:17:19.114979 20650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 02:17:19.115154 20650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 02:17:19.120216 20560 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [term 1 FOLLOWER]: Refusing update from remote peer bda02d487386484d850c91787245f732: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:19.120501 20626 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:19.121665 20606 sys_catalog.cc:455] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bda02d487386484d850c91787245f732. Latest consensus state: current_term: 1 leader_uuid: "bda02d487386484d850c91787245f732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } } }
I20260808 02:17:19.121754 20606 sys_catalog.cc:458] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:19.129297 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.114840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48954 (local address 127.19.58.254:41719)
0808 02:17:19.114874 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.114876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.116975 (+ 2099us) server_negotiation.cc:407] Connection header received
0808 02:17:19.117062 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.117063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.117071 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.117099 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.117702 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.118435 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.122142 (+ 3707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.122453 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.122495 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.125320 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.125321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.125321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.125327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.126821 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.127364 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.127366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.127366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.127422 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.128701 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.128702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.128702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.128844 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.128943 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.129133 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.129207 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 02:17:19.131211 20490 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } attempt: 3
I20260808 02:17:19.134788 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.116114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50612 (local address 127.19.58.252:34919)
0808 02:17:19.116163 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.116164 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.116541 (+ 377us) server_negotiation.cc:407] Connection header received
0808 02:17:19.116629 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.116630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.116639 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.116666 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.119302 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.120047 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.123144 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.123443 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.123482 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.129719 (+ 6237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.129721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.129721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.129728 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.132748 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.133380 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.133381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.133381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.133434 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.133854 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.133855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.133855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.133983 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.134061 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.134626 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.134700 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 02:17:19.135286 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.115864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33128 (local address 127.19.58.253:41665)
0808 02:17:19.115911 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.115913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.116405 (+ 492us) server_negotiation.cc:407] Connection header received
0808 02:17:19.116538 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.116539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.116550 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.116579 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.118455 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.119176 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.123584 (+ 4408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.123913 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.123954 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.130002 (+ 6048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.130004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.130004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.130011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.131668 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.133625 (+ 1957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.133626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.133627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.133676 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.134246 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.134247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.134247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.134377 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.134440 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.135104 (+ 664us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.135176 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 02:17:19.136861 20647 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bda02d487386484d850c91787245f732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } } }
I20260808 02:17:19.136965 20647 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [sys.catalog]: This master's current role is: LEADER
Aug 08 02:17:19 dist-test-slave-1l3l krb5kdc[20242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786155438, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.58.253@KRBTEST.COM for kudu/127.19.58.252@KRBTEST.COM
I20260808 02:17:19.137673 20606 sys_catalog.cc:455] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bda02d487386484d850c91787245f732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } } }
I20260808 02:17:19.137759 20606 sys_catalog.cc:458] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:19.137962 20661 mvcc.cc:205] Tried to move back new op lower bound from 7316092678634033152 to 7316092678604664832. Current Snapshot: MvccSnapshot[applied={T|T < 7316092678634033152}]
I20260808 02:17:19.138090 20663 mvcc.cc:205] Tried to move back new op lower bound from 7316092678634033152 to 7316092678604664832. Current Snapshot: MvccSnapshot[applied={T|T < 7316092678634033152}]
I20260808 02:17:19.140377 20650 catalog_manager.cc:1347] Generated new cluster ID: 289c83eb554044f1b14dc2eb22e81dbb
I20260808 02:17:19.140429 20650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 02:17:19.156701 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.131438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.58.252:34919 (local address 127.0.0.1:50628)
0808 02:17:19.131521 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 02:17:19.131530 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 02:17:19.131565 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 02:17:19.132163 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 02:17:19.132164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 02:17:19.132325 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 02:17:19.132367 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:19.132369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.133937 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.133938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:19.134762 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 02:17:19.134765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.135641 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.135642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 02:17:19.135737 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.136129 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 02:17:19.136135 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 02:17:19.137575 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 02:17:19.140147 (+ 2572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:19.140148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:19.140149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:19.140367 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:19.155928 (+ 15561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 02:17:19.155930 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 02:17:19.155930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 02:17:19.156011 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 02:17:19.156365 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 02:17:19.156366 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 02:17:19.156431 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 02:17:19.156490 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 02:17:19.156599 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 02:17:19.157343 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:19.131659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50628 (local address 127.19.58.252:34919)
0808 02:17:19.131934 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.131936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.131971 (+ 35us) server_negotiation.cc:407] Connection header received
0808 02:17:19.131991 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.131991 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.132000 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.132450 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.132531 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.133819 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.134989 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:19.135533 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:19.138198 (+ 2665us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:19.138244 (+ 46us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 02:17:19.138245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 02:17:19.138245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 02:17:19.138249 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 02:17:19.140020 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.155749 (+ 15729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.155750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.155750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.155842 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 02:17:19.156108 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 02:17:19.156109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 02:17:19.156109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 02:17:19.156265 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 02:17:19.156323 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:19.157158 (+ 835us) server_negotiation.cc:299] Negotiation successful
0808 02:17:19.157248 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":205,"threads_started":1}
I20260808 02:17:19.158806 20490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3: Bootstrap starting.
I20260808 02:17:19.159507 20490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3: Neither blocks nor log segments found. Creating new log.
I20260808 02:17:19.159845 20490 log.cc:824] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3: Log is configured to *not* fsync() on all Append() calls
I20260808 02:17:19.162025 20650 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 02:17:19.163249 20650 catalog_manager.cc:1504] Loading token signing keys...
I20260808 02:17:19.164345 20490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3: No bootstrap required, opened a new log
I20260808 02:17:19.166293 20490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } }
I20260808 02:17:19.166438 20490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29a113b6c091496dbe82e12880e5dbc3, State: Initialized, Role: FOLLOWER
I20260808 02:17:19.166519 20490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } }
I20260808 02:17:19.166869 20490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 02:17:19.166980 20449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 02:17:19.167618 20453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bda02d487386484d850c91787245f732" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "29a113b6c091496dbe82e12880e5dbc3"
I20260808 02:17:19.168597 20449 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [term 1 FOLLOWER]: Refusing update from remote peer bda02d487386484d850c91787245f732: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 02:17:19.169754 20453 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bda02d487386484d850c91787245f732 in term 1.
I20260808 02:17:19.170235 20647 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 02:17:19.172569 20691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bda02d487386484d850c91787245f732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } } }
I20260808 02:17:19.172660 20691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:19.172717 20691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bda02d487386484d850c91787245f732. Latest consensus state: current_term: 1 leader_uuid: "bda02d487386484d850c91787245f732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } } }
I20260808 02:17:19.172772 20691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:19.172817 20691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bda02d487386484d850c91787245f732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } } }
I20260808 02:17:19.172864 20691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 02:17:19.173732 20626 sys_catalog.cc:455] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bda02d487386484d850c91787245f732" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bda02d487386484d850c91787245f732" member_type: VOTER last_known_addr { host: "127.19.58.254" port: 41719 } } peers { permanent_uuid: "29a113b6c091496dbe82e12880e5dbc3" member_type: VOTER last_known_addr { host: "127.19.58.253" port: 41665 } } peers { permanent_uuid: "d896c1df18424ac6bb91f847f3d29cb3" member_type: VOTER last_known_addr { host: "127.19.58.252" port: 34919 } } }
I20260808 02:17:19.173830 20626 sys_catalog.cc:458] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732 [sys.catalog]: This master's current role is: LEADER
I20260808 02:17:19.173496 20689 mvcc.cc:205] Tried to move back new op lower bound from 7316092678803681280 to 7316092678604664832. Current Snapshot: MvccSnapshot[applied={T|T < 7316092678634033152}]
W20260808 02:17:19.175150 20686 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 02:17:19.175207 20686 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 02:17:19.175287 20650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bda02d487386484d850c91787245f732: Generated new TSK 0
I20260808 02:17:19.175464 20650 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 02:17:19.204773 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.203170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33134 (local address 127.19.58.253:41665)
0808 02:17:19.203215 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.203217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.204111 (+ 894us) server_negotiation.cc:407] Connection header received
0808 02:17:19.204215 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.204216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.204227 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.204257 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.204707 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33134: BlockingRecv error: recv got EOF from 127.0.0.1:33134 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 02:17:19.205672 20380 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.203893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48964 (local address 127.19.58.254:41719)
0808 02:17:19.203938 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.203940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.205170 (+ 1230us) server_negotiation.cc:407] Connection header received
0808 02:17:19.205264 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.205265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.205274 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.205303 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.205611 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48964: BlockingRecv error: recv got EOF from 127.0.0.1:48964 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 02:17:19.206228 20665 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.203534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50630 (local address 127.19.58.252:34919)
0808 02:17:19.203589 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.203591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.205010 (+ 1419us) server_negotiation.cc:407] Connection header received
0808 02:17:19.205046 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.205047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.205058 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.205122 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.206169 (+ 1047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50630: BlockingRecv error: recv got EOF from 127.0.0.1:50630 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 02:17:19.208506 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.207753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.19.58.253:41665)
0808 02:17:19.207790 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.207791 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.208040 (+ 249us) server_negotiation.cc:407] Connection header received
0808 02:17:19.208136 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.208137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.208153 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.208181 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.208462 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33146: BlockingRecv error: recv got EOF from 127.0.0.1:33146 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 02:17:19.211513 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.210758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33154 (local address 127.19.58.253:41665)
0808 02:17:19.210797 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.210798 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.211048 (+ 250us) server_negotiation.cc:407] Connection header received
0808 02:17:19.211139 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.211140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.211150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.211178 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.211474 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33154: BlockingRecv error: recv got EOF from 127.0.0.1:33154 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 02:17:19.215569 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.214807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.19.58.253:41665)
0808 02:17:19.214838 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.214840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.215082 (+ 242us) server_negotiation.cc:407] Connection header received
0808 02:17:19.215172 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.215173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.215183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.215211 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.215532 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33160: BlockingRecv error: recv got EOF from 127.0.0.1:33160 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 02:17:19.221426 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.220642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33164 (local address 127.19.58.253:41665)
0808 02:17:19.220675 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.220676 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.220931 (+ 255us) server_negotiation.cc:407] Connection header received
0808 02:17:19.221031 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.221032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.221042 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.221070 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.221385 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33164: BlockingRecv error: recv got EOF from 127.0.0.1:33164 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 02:17:19.231349 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.230521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33174 (local address 127.19.58.253:41665)
0808 02:17:19.230555 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.230557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.230844 (+ 287us) server_negotiation.cc:407] Connection header received
0808 02:17:19.230941 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.230942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.230953 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.230982 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.231294 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33174: BlockingRecv error: recv got EOF from 127.0.0.1:33174 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 02:17:19.249423 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.248558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33178 (local address 127.19.58.253:41665)
0808 02:17:19.248596 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.248598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.248874 (+ 276us) server_negotiation.cc:407] Connection header received
0808 02:17:19.248977 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.248978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.248990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.249020 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.249374 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33178: BlockingRecv error: recv got EOF from 127.0.0.1:33178 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 02:17:19.283408 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.282749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33194 (local address 127.19.58.253:41665)
0808 02:17:19.282804 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.282806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.282878 (+ 72us) server_negotiation.cc:407] Connection header received
0808 02:17:19.282931 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.282933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.282946 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.283021 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.283343 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33194: BlockingRecv error: recv got EOF from 127.0.0.1:33194 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260808 02:17:19.349126 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.348452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.19.58.253:41665)
0808 02:17:19.348529 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.348531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.348596 (+ 65us) server_negotiation.cc:407] Connection header received
0808 02:17:19.348642 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.348643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.348654 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.348715 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.349079 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33208: BlockingRecv error: recv got EOF from 127.0.0.1:33208 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 02:17:19.479084 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.478323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33216 (local address 127.19.58.253:41665)
0808 02:17:19.478432 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.478434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.478601 (+ 167us) server_negotiation.cc:407] Connection header received
0808 02:17:19.478635 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.478636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.478648 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.478701 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.479014 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33216: BlockingRecv error: recv got EOF from 127.0.0.1:33216 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 02:17:19.736825 20495 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 02:17:19.736236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33220 (local address 127.19.58.253:41665)
0808 02:17:19.736299 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:19.736301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:19.736319 (+ 18us) server_negotiation.cc:407] Connection header received
0808 02:17:19.736394 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:19.736395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:19.736406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:19.736444 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 02:17:19.736764 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33220: BlockingRecv error: recv got EOF from 127.0.0.1:33220 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 02:17:20.090167 20623 catalog_manager.cc:1259] Loaded cluster ID: 289c83eb554044f1b14dc2eb22e81dbb
I20260808 02:17:20.090257 20623 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3: loading cluster ID for follower catalog manager: success
I20260808 02:17:20.091109 20623 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3: acquiring CA information for follower catalog manager: success
I20260808 02:17:20.091408 20623 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d896c1df18424ac6bb91f847f3d29cb3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:17:20.175642 20686 catalog_manager.cc:1259] Loaded cluster ID: 289c83eb554044f1b14dc2eb22e81dbb
I20260808 02:17:20.175741 20686 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3: loading cluster ID for follower catalog manager: success
I20260808 02:17:20.176699 20686 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3: acquiring CA information for follower catalog manager: success
I20260808 02:17:20.177058 20686 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 29a113b6c091496dbe82e12880e5dbc3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 02:17:20.252154 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:20.249828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.19.58.253:41665)
0808 02:17:20.249925 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:20.249927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:20.249980 (+ 53us) server_negotiation.cc:407] Connection header received
0808 02:17:20.250038 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:20.250039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:20.250087 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:20.250147 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:20.250262 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:20.250752 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:20.251459 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:20.251652 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:20.251724 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:20.251854 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:20.251967 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:20.252023 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:20.252053 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 02:17:20.252070 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":49,"threads_started":1}
I20260808 02:17:20.255869 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:20.253814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.19.58.252:34919)
0808 02:17:20.253931 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:20.253932 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:20.253940 (+ 8us) server_negotiation.cc:407] Connection header received
0808 02:17:20.253978 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:20.253978 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:20.254012 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:20.254078 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:20.254181 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:20.254640 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:20.255275 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:20.255468 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:20.255512 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:20.255602 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:20.255686 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:20.255777 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:20.255791 (+ 14us) server_negotiation.cc:299] Negotiation successful
0808 02:17:20.255809 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260808 02:17:20.260142 20816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:20.257323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48972 (local address 127.19.58.254:41719)
0808 02:17:20.257427 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:20.257428 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:20.257440 (+ 12us) server_negotiation.cc:407] Connection header received
0808 02:17:20.257542 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:20.257544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:20.257596 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:20.257667 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:20.257744 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:20.258316 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:20.259255 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:20.259538 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:20.259591 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:20.259795 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:20.259891 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:20.259990 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:20.260037 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 02:17:20.260058 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":50,"threads_started":1}
I20260808 02:17:20.260324 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:20.257333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33244 (local address 127.19.58.253:41665)
0808 02:17:20.257367 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:20.257369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:20.257587 (+ 218us) server_negotiation.cc:407] Connection header received
0808 02:17:20.257639 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:20.257640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:20.257651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:20.257705 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:20.257854 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:20.258535 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:20.259549 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:20.259867 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:20.259932 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:20.260068 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:20.260101 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:20.260129 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:20.260231 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 02:17:20.260254 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 02:17:20.260457 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 02:17:20.257580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50646 (local address 127.19.58.252:34919)
0808 02:17:20.257773 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 02:17:20.257775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 02:17:20.257782 (+ 7us) server_negotiation.cc:407] Connection header received
0808 02:17:20.257803 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 02:17:20.257804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 02:17:20.257812 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 02:17:20.257855 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 02:17:20.258008 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:20.258679 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:20.259684 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 02:17:20.259920 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 02:17:20.259973 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 02:17:20.260130 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 02:17:20.260159 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 02:17:20.260310 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 02:17:20.260373 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 02:17:20.260402 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260808 02:17:20.261880 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 20265
I20260808 02:17:20.271420 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 20384
I20260808 02:17:20.284818 19691 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDIPgti/build/release/bin/kudu with pid 20498
2026-08-08T02:17:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2869 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5732 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (269750 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST