Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Create Nominal Test MR To Check Bot #1411

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

WIP: Create Nominal Test MR To Check Bot #1411

wants to merge 4 commits into from

Conversation

Cynical-Optimist
Copy link

See original merge request on GitLab
In GitLab by [Gitlab user @lchlan] on Dec 13, 2019, 15:11

Description

This is designed to check the bots behavior by adding a delay in.

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @buildstream-benchmarking-bot] on Dec 13, 2019, 16:48

Benchmarking request accepted: 2019-12-13 16:48:38.412352
Benchmark run accepted, pipeline link is: https://gitlab.com/BuildStream/benchmarking/benchmarks/pipelines/102959634
Benchmark run success: Pipeline 102932641
Benchmarking started at 2019-12-13T16:48:40.179Z, complete at 2019-12-13T19:07:45.808Z

Time SHA Results File Average Resident Set Size (kb) Test Version Time Standard Deviation Result Date Job ID Error Type Error Message Test Name Average Time to Complete Test (s) RSS Standard Deviation
2019-12-13 17:12:23.873860 82e03a0e4 results_out/results.json 98954.66666666667 lachlanmackenzie/CheckBotBehaviour 0.24853382879482694 2019-12-13 12:47:42 benchmarks Build of Baserock stage1-binutils for x86_64 80.24817951520284 7437.634390942682
2019-12-13 17:12:23.873860 2fc5a5c results_out/results.json 103685.33333333333 master 1.1846920846313151 2019-12-13 17:12:23.873860 benchmarks Build of Baserock stage1-binutils for x86_64 80.68290797869365 85.54141297250901
results_out/results.json stable ERROR 20191213-171224 benchmarks missing Measurement Missing Build of Baserock stage1-binutils for x86_64 ERROR
2019-12-13 18:52:29.713877 82e03a0e4 results_out/results.json 627650.6666666666 lachlanmackenzie/CheckBotBehaviour 0.30659157775748874 2019-12-13 12:47:42 test-bstgen-100-square Hundred Square bstgen 77.57001455624898 8451.2459042045
2019-12-13 18:52:29.713877 2fc5a5c results_out/results.json 651354.6666666666 master 0.4605528376734667 2019-12-13 18:52:29.713877 test-bstgen-100-square Hundred Square bstgen 77.60956573486328 64.29100507328637
2019-12-13 17:50:27.008988 82e03a0e4 results_out/results.json 1319880 lachlanmackenzie/CheckBotBehaviour 0.6823847273220625 2019-12-13 12:47:42 test-debian-like-project Show of Debian like project for x86_64 159.85273369153342 4011.6290955171817
2019-12-13 17:50:27.008988 2fc5a5c results_out/results.json 1318808 master 0.8557925946666037 2019-12-13 17:50:27.008988 test-debian-like-project Show of Debian like project for x86_64 158.563206354777 1752.0547936637142
2019-12-13 17:12:23.873860 82e03a0e4 results_out/results.json 37986.666666666664 lachlanmackenzie/CheckBotBehaviour 0.11855746452188791 2019-12-13 12:47:42 benchmarks Startup time 0.38839133580525714 34.01960219246153
2019-12-13 17:12:23.873860 2fc5a5c results_out/results.json 38222.666666666664 master 0.020240632496197328 2019-12-13 17:12:23.873860 benchmarks Startup time 0.31164948145548504 119.48779575058423
2019-12-13 17:12:23.873860 fe9aeb8 results_out/results.json 23272 stable 0.038806291934436854 2019-12-13 17:12:23.873860 benchmarks Startup time 0.20747772852579752 10.583005244258363
2019-12-13 18:23:02.492534 82e03a0e4 results_out/results.json 38230.666666666664 lachlanmackenzie/CheckBotBehaviour 0.03575996662543305 2019-12-13 12:47:42 functional-test-combined-config Startup time 0.3435189723968506 185.44361227427956
2019-12-13 18:23:02.492534 2fc5a5c results_out/results.json 38322.666666666664 master 0.008425726082781474 2019-12-13 18:23:02.492534 functional-test-combined-config Startup time 0.29974929491678876 210.4978226332361
2019-12-13 18:23:02.492534 fe9aeb8 results_out/results.json 23400 stable 0.03125414355171663 2019-12-13 18:23:02.492534 functional-test-combined-config Startup time 0.18973700205485025 67.29041536504289
2019-12-13 18:05:24.986856 82e03a0e4 results_out/results.json 38188 lachlanmackenzie/CheckBotBehaviour 0.02131073659417826 2019-12-13 12:47:42 functional-test-simple Startup time 0.3109890619913737 217.14511277023942
2019-12-13 18:05:24.986856 2fc5a5c results_out/results.json 38281.333333333336 master 0.041387918560079145 2019-12-13 18:05:24.986856 functional-test-simple Startup time 0.33248122533162433 40.85747585611883

Digest of results can be found at: https://gitlab.com/BuildStream/benchmarking/benchmarks/-/jobs/378996858/artifacts/raw/pipeline_cache/all_results_digest.csv
Benchmarking remove labels MR: 45057829 in response to Benchmarking pipeline 102959634

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Dec 13, 2019, 20:43

added 2 commits

  • b6d9af12 - Add nominal del for version
  • d1d7e46b - Add delay into app

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Dec 16, 2019, 14:33

added 1 commit

  • 87ad5363 - Add delay into app

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Dec 16, 2019, 18:42

added 2 commits

  • 7d5b62c1 - Add delay into app
  • b1e11949 - Increase timeout to prevent delays causing test failures

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Dec 18, 2019, 10:47

added 26 commits

  • b1e11949...d1d898bd - 23 commits from branch master
  • 5e15ccf6 - Add nominal del for version
  • 86375235 - Add delay into app
  • 2bcbf28d - Increase timeout to prevent delays causing test failures

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Dec 19, 2019, 11:49

added 5 commits

  • 2bcbf28d...8ec01832 - 2 commits from branch master
  • 03740cee - Add nominal del for version
  • 14507292 - Add delay into app
  • 545f4962 - Increase timeout to prevent delays causing test failures

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Dec 20, 2019, 12:04

added 5 commits

  • 545f4962...bb93d63f - 2 commits from branch master
  • ddc47162 - Add nominal del for version
  • da54473d - Add delay into app
  • 92eea45d - Increase timeout to prevent delays causing test failures

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Dec 23, 2019, 15:02

added 8 commits

  • 92eea45d...0bc77c68 - 4 commits from branch master
  • 6ccb2988 - Add nominal del for version
  • b80f469c - Add delay into app
  • bed329cd - Increase timeout to prevent delays causing test failures
  • 70587e53 - Merge branch 'lachlanmackenzie/CheckBotBehaviour' of...

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Dec 24, 2019, 18:28

added 17 commits

  • 70587e53...55bf729c - 14 commits from branch master
  • 41365903 - Add nominal del for version
  • 75faac75 - Add delay into app
  • 52092a03 - Increase timeout to prevent delays causing test failures

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Dec 31, 2019, 18:52

added 15 commits

  • 52092a03...c8153bc7 - 12 commits from branch master
  • 6039ca1d - Add nominal del for version
  • f36f185e - Add delay into app
  • f62b3d7a - Increase timeout to prevent delays causing test failures

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Jan 2, 2020, 16:54

added 7 commits

  • f62b3d7a...51263e21 - 4 commits from branch master
  • fcce78de - Add nominal del for version
  • 9bf1bdf9 - Add delay into app
  • 3f6fc4e0 - Increase timeout to prevent delays causing test failures

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Jan 10, 2020, 10:51

added 12 commits

  • 3f6fc4e0...928d8e6b - 9 commits from branch master
  • 85641efa - Add nominal del for version
  • 3d5f7724 - Add delay into app
  • d53637a5 - Increase timeout to prevent delays causing test failures

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Jan 10, 2020, 15:56

added 1 commit

  • 173be100 - Revert "Add nominal del for version"

Compare with previous version

@Cynical-Optimist
Copy link
Author

In GitLab by [Gitlab user @lchlan] on Feb 10, 2020, 12:07

added 103 commits

  • 173be100...0591b0c8 - 99 commits from branch master
  • 255630f - Add nominal del for version
  • 66f0974 - Add delay into app
  • 6386122 - Increase timeout to prevent delays causing test failures
  • e11fd57 - Revert "Add nominal del for version"

Compare with previous version

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
benchmark_prio-1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Cynical-Optimist