Diagnosed failure
MaintenanceManagerTest.TestPriorityOpLaunch: /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/maintenance_manager-test.cc:867: Failure
Expected equality of these values:
ordered_ops.front()
Which is: "op6"
instance.name()
Which is: "op7"
I20260201 15:00:23.380772 11376 test_util.cc:183] -----------------------------------------------
I20260201 15:00:23.380990 11376 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskui6oFp/test-tmp/maintenance_manager-test.0.MaintenanceManagerTest.TestPriorityOpLaunch.1769958021015915-11376-0
Full log
[==========] Running 16 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 16 tests from MaintenanceManagerTest
[ RUN ] MaintenanceManagerTest.TestCreateAndShutdown
[ OK ] MaintenanceManagerTest.TestCreateAndShutdown (10 ms)
[ RUN ] MaintenanceManagerTest.TestRegisterUnregister
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260201 15:00:21.048609 11387 maintenance_manager-test.cc:131] Prepared op 1
I20260201 15:00:21.049115 11387 maintenance_manager.cc:419] P 12345: Scheduling 1: perf score=10.000000
I20260201 15:00:21.050132 11386 maintenance_manager-test.cc:138] Performing op 1
I20260201 15:00:21.050509 11386 maintenance_manager.cc:643] P 12345: 1 complete. Timing: real 0.000s user 0.001s sys 0.000s Metrics: {}
[ OK ] MaintenanceManagerTest.TestRegisterUnregister (11 ms)
[ RUN ] MaintenanceManagerTest.TestRegisterUnregisterWithContention
[ OK ] MaintenanceManagerTest.TestRegisterUnregisterWithContention (7 ms)
[ RUN ] MaintenanceManagerTest.TestNewOpsDontGetScheduledDuringUnregister
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/maintenance_manager-test.cc:453: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
[ SKIPPED ] MaintenanceManagerTest.TestNewOpsDontGetScheduledDuringUnregister (6 ms)
[ RUN ] MaintenanceManagerTest.TestMemoryPressurePrioritizesMemory
I20260201 15:00:21.093192 11397 maintenance_manager-test.cc:131] Prepared op op
I20260201 15:00:21.093379 11397 maintenance_manager.cc:419] P 12345: Scheduling op: under memory pressure (0.00% used), 0 bytes log retention, and flush 100 bytes memory
I20260201 15:00:21.094105 11396 maintenance_manager-test.cc:138] Performing op op
I20260201 15:00:21.094455 11396 maintenance_manager.cc:643] P 12345: op complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
[ OK ] MaintenanceManagerTest.TestMemoryPressurePrioritizesMemory (30 ms)
[ RUN ] MaintenanceManagerTest.TestMemoryPressurePerformsNoMemoryOp
I20260201 15:00:21.123526 11400 maintenance_manager-test.cc:131] Prepared op op
I20260201 15:00:21.123732 11400 maintenance_manager.cc:419] P 12345: Scheduling op: perf score=1.000000
I20260201 15:00:21.124611 11399 maintenance_manager-test.cc:138] Performing op op
I20260201 15:00:21.124943 11399 maintenance_manager.cc:643] P 12345: op complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
[ OK ] MaintenanceManagerTest.TestMemoryPressurePerformsNoMemoryOp (29 ms)
[ RUN ] MaintenanceManagerTest.TestLogRetentionPrioritization
[ OK ] MaintenanceManagerTest.TestLogRetentionPrioritization (5 ms)
[ RUN ] MaintenanceManagerTest.TestPrioritizeLogRetentionUnderMemoryPressure
[ OK ] MaintenanceManagerTest.TestPrioritizeLogRetentionUnderMemoryPressure (5 ms)
[ RUN ] MaintenanceManagerTest.RunningInstances
I20260201 15:00:21.143805 11409 maintenance_manager-test.cc:131] Prepared op op
I20260201 15:00:21.144025 11409 maintenance_manager.cc:419] P 12345: Scheduling op: perf score=10.000000
I20260201 15:00:21.144536 11409 maintenance_manager-test.cc:131] Prepared op op
I20260201 15:00:21.144716 11409 maintenance_manager.cc:419] P 12345: Scheduling op: perf score=10.000000
I20260201 15:00:21.144726 11408 maintenance_manager-test.cc:138] Performing op op
I20260201 15:00:22.282958 11407 maintenance_manager-test.cc:138] Performing op op
I20260201 15:00:23.283181 11408 maintenance_manager.cc:643] P 12345: op complete. Timing: real 2.138s user 0.005s sys 0.006s Metrics: {"spinlock_wait_cycles":89856}
I20260201 15:00:23.285532 11407 maintenance_manager.cc:643] P 12345: op complete. Timing: real 2.140s user 0.010s sys 0.005s Metrics: {"spinlock_wait_cycles":2615941504}
[ OK ] MaintenanceManagerTest.RunningInstances (2149 ms)
[ RUN ] MaintenanceManagerTest.TestCompletedOpsHistory
I20260201 15:00:23.293924 11413 maintenance_manager-test.cc:131] Prepared op op0
I20260201 15:00:23.294142 11413 maintenance_manager.cc:419] P 12345: Scheduling op0: perf score=1.000000
I20260201 15:00:23.294806 11412 maintenance_manager-test.cc:138] Performing op op0
I20260201 15:00:23.295086 11412 maintenance_manager.cc:643] P 12345: op0 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.297950 11413 maintenance_manager-test.cc:131] Prepared op op1
I20260201 15:00:23.298111 11413 maintenance_manager.cc:419] P 12345: Scheduling op1: perf score=1.000000
I20260201 15:00:23.298616 11412 maintenance_manager-test.cc:138] Performing op op1
I20260201 15:00:23.299065 11412 maintenance_manager.cc:643] P 12345: op1 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.301942 11413 maintenance_manager-test.cc:131] Prepared op op2
I20260201 15:00:23.302100 11413 maintenance_manager.cc:419] P 12345: Scheduling op2: perf score=1.000000
I20260201 15:00:23.302620 11412 maintenance_manager-test.cc:138] Performing op op2
I20260201 15:00:23.302925 11412 maintenance_manager.cc:643] P 12345: op2 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.306968 11413 maintenance_manager-test.cc:131] Prepared op op3
I20260201 15:00:23.307229 11413 maintenance_manager.cc:419] P 12345: Scheduling op3: perf score=1.000000
I20260201 15:00:23.307760 11412 maintenance_manager-test.cc:138] Performing op op3
I20260201 15:00:23.308027 11412 maintenance_manager.cc:643] P 12345: op3 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.310738 11413 maintenance_manager-test.cc:131] Prepared op op4
I20260201 15:00:23.310892 11413 maintenance_manager.cc:419] P 12345: Scheduling op4: perf score=1.000000
I20260201 15:00:23.311429 11412 maintenance_manager-test.cc:138] Performing op op4
I20260201 15:00:23.311697 11412 maintenance_manager.cc:643] P 12345: op4 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.314668 11413 maintenance_manager-test.cc:131] Prepared op op5
I20260201 15:00:23.314888 11413 maintenance_manager.cc:419] P 12345: Scheduling op5: perf score=1.000000
I20260201 15:00:23.315478 11412 maintenance_manager-test.cc:138] Performing op op5
I20260201 15:00:23.315797 11412 maintenance_manager.cc:643] P 12345: op5 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.319890 11413 maintenance_manager-test.cc:131] Prepared op op6
I20260201 15:00:23.320055 11413 maintenance_manager.cc:419] P 12345: Scheduling op6: perf score=1.000000
I20260201 15:00:23.320667 11412 maintenance_manager-test.cc:138] Performing op op6
I20260201 15:00:23.320961 11412 maintenance_manager.cc:643] P 12345: op6 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.323726 11413 maintenance_manager-test.cc:131] Prepared op op7
I20260201 15:00:23.323916 11413 maintenance_manager.cc:419] P 12345: Scheduling op7: perf score=1.000000
I20260201 15:00:23.324451 11412 maintenance_manager-test.cc:138] Performing op op7
I20260201 15:00:23.324797 11412 maintenance_manager.cc:643] P 12345: op7 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.327651 11413 maintenance_manager-test.cc:131] Prepared op op8
I20260201 15:00:23.327818 11413 maintenance_manager.cc:419] P 12345: Scheduling op8: perf score=1.000000
I20260201 15:00:23.328286 11412 maintenance_manager-test.cc:138] Performing op op8
I20260201 15:00:23.328588 11412 maintenance_manager.cc:643] P 12345: op8 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.332679 11413 maintenance_manager-test.cc:131] Prepared op op9
I20260201 15:00:23.332849 11413 maintenance_manager.cc:419] P 12345: Scheduling op9: perf score=1.000000
I20260201 15:00:23.333519 11412 maintenance_manager-test.cc:138] Performing op op9
I20260201 15:00:23.333815 11412 maintenance_manager.cc:643] P 12345: op9 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.336627 11413 maintenance_manager-test.cc:131] Prepared op op10
I20260201 15:00:23.336822 11413 maintenance_manager.cc:419] P 12345: Scheduling op10: perf score=1.000000
I20260201 15:00:23.337349 11412 maintenance_manager-test.cc:138] Performing op op10
I20260201 15:00:23.337653 11412 maintenance_manager.cc:643] P 12345: op10 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
[ OK ] MaintenanceManagerTest.TestCompletedOpsHistory (53 ms)
[ RUN ] MaintenanceManagerTest.TestOpFactors
[ OK ] MaintenanceManagerTest.TestOpFactors (5 ms)
[ RUN ] MaintenanceManagerTest.TestPriorityOpLaunch
I20260201 15:00:23.353735 11421 maintenance_manager-test.cc:131] Prepared op canary
I20260201 15:00:23.353992 11421 maintenance_manager.cc:419] P 12345: Scheduling canary: perf score=1.000000
I20260201 15:00:23.354619 11420 maintenance_manager-test.cc:138] Performing op canary
I20260201 15:00:23.354913 11420 maintenance_manager.cc:643] P 12345: canary complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.360504 11421 maintenance_manager-test.cc:131] Prepared op op6
I20260201 15:00:23.360751 11421 maintenance_manager.cc:419] P 12345: Scheduling op6: perf score=16.105100
I20260201 15:00:23.361342 11420 maintenance_manager-test.cc:138] Performing op op6
I20260201 15:00:23.362615 11420 maintenance_manager.cc:643] P 12345: op6 complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.363195 11421 maintenance_manager-test.cc:131] Prepared op op7
I20260201 15:00:23.363345 11421 maintenance_manager.cc:419] P 12345: Scheduling op7: perf score=19.000000
I20260201 15:00:23.363791 11420 maintenance_manager-test.cc:138] Performing op op7
I20260201 15:00:23.365067 11420 maintenance_manager.cc:643] P 12345: op7 complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.365598 11421 maintenance_manager-test.cc:131] Prepared op op5
I20260201 15:00:23.365744 11421 maintenance_manager.cc:419] P 12345: Scheduling op5: perf score=12.000000
I20260201 15:00:23.366195 11420 maintenance_manager-test.cc:138] Performing op op5
I20260201 15:00:23.367450 11420 maintenance_manager.cc:643] P 12345: op5 complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.367960 11421 maintenance_manager-test.cc:131] Prepared op op4
I20260201 15:00:23.368109 11421 maintenance_manager.cc:419] P 12345: Scheduling op4: perf score=11.000000
I20260201 15:00:23.368561 11420 maintenance_manager-test.cc:138] Performing op op4
I20260201 15:00:23.369815 11420 maintenance_manager.cc:643] P 12345: op4 complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.370328 11421 maintenance_manager-test.cc:131] Prepared op op3
I20260201 15:00:23.370478 11421 maintenance_manager.cc:419] P 12345: Scheduling op3: perf score=10.000000
I20260201 15:00:23.370903 11420 maintenance_manager-test.cc:138] Performing op op3
I20260201 15:00:23.372219 11420 maintenance_manager.cc:643] P 12345: op3 complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.372704 11421 maintenance_manager-test.cc:131] Prepared op op2
I20260201 15:00:23.372854 11421 maintenance_manager.cc:419] P 12345: Scheduling op2: perf score=9.090909
I20260201 15:00:23.373304 11420 maintenance_manager-test.cc:138] Performing op op2
I20260201 15:00:23.374555 11420 maintenance_manager.cc:643] P 12345: op2 complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.375128 11421 maintenance_manager-test.cc:131] Prepared op op1
I20260201 15:00:23.375274 11421 maintenance_manager.cc:419] P 12345: Scheduling op1: perf score=6.209213
I20260201 15:00:23.375710 11420 maintenance_manager-test.cc:138] Performing op op1
I20260201 15:00:23.376971 11420 maintenance_manager.cc:643] P 12345: op1 complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {}
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/maintenance_manager-test.cc:867: Failure
Expected equality of these values:
ordered_ops.front()
Which is: "op6"
instance.name()
Which is: "op7"
I20260201 15:00:23.380772 11376 test_util.cc:183] -----------------------------------------------
I20260201 15:00:23.380990 11376 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskui6oFp/test-tmp/maintenance_manager-test.0.MaintenanceManagerTest.TestPriorityOpLaunch.1769958021015915-11376-0
[ FAILED ] MaintenanceManagerTest.TestPriorityOpLaunch (33 ms)
[ RUN ] MaintenanceManagerTest.VerifyMetrics
I20260201 15:00:23.395545 11424 maintenance_manager.cc:411] P 12345: Prepare failed for oppf. Re-running scheduler.
I20260201 15:00:23.406059 11424 maintenance_manager-test.cc:131] Prepared op op0
I20260201 15:00:23.406266 11424 maintenance_manager.cc:419] P 12345: Scheduling op0: perf score=4.000000
I20260201 15:00:23.406965 11423 maintenance_manager-test.cc:138] Performing op op0
I20260201 15:00:23.407406 11423 maintenance_manager.cc:643] P 12345: op0 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.416947 11424 maintenance_manager-test.cc:131] Prepared op op1
I20260201 15:00:23.417196 11424 maintenance_manager.cc:419] P 12345: Scheduling op1: perf score=2.000000
I20260201 15:00:23.417752 11423 maintenance_manager-test.cc:138] Performing op op1
I20260201 15:00:23.418094 11423 maintenance_manager.cc:643] P 12345: op1 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
[ OK ] MaintenanceManagerTest.VerifyMetrics (49 ms)
[ RUN ] MaintenanceManagerTest.ManyOperationsHeavyUpdateStats
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/maintenance_manager-test.cc:949: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
[ SKIPPED ] MaintenanceManagerTest.ManyOperationsHeavyUpdateStats (5 ms)
[ RUN ] MaintenanceManagerTest.TestUnregisterWhileScheduling
I20260201 15:00:23.439486 11430 maintenance_manager-test.cc:131] Prepared op 1
I20260201 15:00:23.439682 11430 maintenance_manager.cc:419] P 12345: Scheduling 1: perf score=10.000000
I20260201 15:00:23.440279 11430 maintenance_manager-test.cc:131] Prepared op 1
I20260201 15:00:23.440569 11430 maintenance_manager.cc:419] P 12345: Scheduling 1: perf score=10.000000
I20260201 15:00:23.440627 11429 maintenance_manager-test.cc:138] Performing op 1
I20260201 15:00:23.441320 11428 maintenance_manager-test.cc:138] Performing op 1
I20260201 15:00:23.441201 11429 maintenance_manager.cc:643] P 12345: 1 complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"spinlock_wait_cycles":621824}
I20260201 15:00:23.441642 11428 maintenance_manager.cc:643] P 12345: 1 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
I20260201 15:00:23.441958 11430 maintenance_manager-test.cc:131] Prepared op 1
I20260201 15:00:23.442165 11430 maintenance_manager.cc:419] P 12345: Scheduling 1: perf score=10.000000
I20260201 15:00:23.442636 11428 maintenance_manager-test.cc:138] Performing op 1
I20260201 15:00:23.442921 11428 maintenance_manager.cc:643] P 12345: 1 complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {}
[ OK ] MaintenanceManagerTest.TestUnregisterWhileScheduling (9 ms)
[ RUN ] MaintenanceManagerTest.ComprehensiveTest
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/maintenance_manager-test.cc:1032: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
[ SKIPPED ] MaintenanceManagerTest.ComprehensiveTest (4 ms)
[----------] 16 tests from MaintenanceManagerTest (2419 ms total)
[----------] Global test environment tear-down
[==========] 16 tests from 1 test suite ran. (2420 ms total)
[ PASSED ] 12 tests.
[ SKIPPED ] 3 tests, listed below:
[ SKIPPED ] MaintenanceManagerTest.TestNewOpsDontGetScheduledDuringUnregister
[ SKIPPED ] MaintenanceManagerTest.ManyOperationsHeavyUpdateStats
[ SKIPPED ] MaintenanceManagerTest.ComprehensiveTest
[ FAILED ] 1 test, listed below:
[ FAILED ] MaintenanceManagerTest.TestPriorityOpLaunch
1 FAILED TEST