Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark parser on CI #137

Open
keynmol opened this issue Jan 12, 2023 · 0 comments
Open

Benchmark parser on CI #137

keynmol opened this issue Jan 12, 2023 · 0 comments
Labels

Comments

@keynmol
Copy link
Collaborator

keynmol commented Jan 12, 2023

To make sure we're not regressing the parser, I suggest we run a smoke test for performance on PRs.

Basically as simple as

hyperfine 'tree-sitter parse scala/src/library/**/*.scala --quiet --time --stat'

I've run it on several past commits and it seems to be pretty stable:

HEAD is now at 47767fc Merge pull request #135 from ckipp01/generationThrice

Time (mean ± σ):      40.8 ms ±   1.0 ms    [User: 39.4 ms, System: 1.4 ms]
Range (min … max):    39.6 ms …  44.8 ms    71 runs

HEAD is now at 9cf2764 chore: generate latest changes

Time (mean ± σ):      41.1 ms ±   1.5 ms    [User: 39.4 ms, System: 1.8 ms]
Range (min … max):    39.5 ms …  48.5 ms    71 runs

HEAD is now at fd05f09 Merge pull request #131 from eed3si9n/wip/instance

Time (mean ± σ):      41.0 ms ±   1.6 ms    [User: 39.1 ms, System: 1.9 ms]
Range (min … max):    39.6 ms …  49.8 ms    71 runs

HEAD is now at 14a22e7 Add more test for instance_expression

Time (mean ± σ):      40.3 ms ±   0.6 ms    [User: 38.4 ms, System: 1.9 ms]
Range (min … max):    39.4 ms …  43.5 ms    72 runs

HEAD is now at 77ea27e Fixes instance_expression

Time (mean ± σ):      40.3 ms ±   0.6 ms    [User: 38.1 ms, System: 2.2 ms]
Range (min … max):    39.3 ms …  43.5 ms    72 runs

HEAD is now at 8677ff4 Creates a dummy node for import_selectors (#126)

Time (mean ± σ):      41.4 ms ±   5.9 ms    [User: 39.2 ms, System: 1.6 ms]
Range (min … max):    39.9 ms …  90.2 ms    71 runs

HEAD is now at 94bb82c Merge pull request #125 from keynmol/export-clauses

Time (mean ± σ):      41.0 ms ±   3.2 ms    [User: 38.8 ms, System: 1.8 ms]
Range (min … max):    39.9 ms …  67.1 ms    71 runs

HEAD is now at accc6c0 Output expected % regardless

Time (mean ± σ):      40.6 ms ±   0.8 ms    [User: 39.1 ms, System: 1.6 ms]
Range (min … max):    39.6 ms …  44.5 ms    72 runs

HEAD is now at 94b7758 Export clauses

Time (mean ± σ):      40.7 ms ±   0.6 ms    [User: 38.6 ms, System: 2.2 ms]
Range (min … max):    39.6 ms …  42.3 ms    70 runs

HEAD is now at 7d7fd07 Merge pull request #124 from eed3si9n/wip/guard

Time (mean ± σ):      40.6 ms ±   0.3 ms    [User: 38.8 ms, System: 1.7 ms]
Range (min … max):    39.9 ms …  41.6 ms    71 runs

HEAD is now at 5b9b24d Merge pull request #122 from ckipp01/generateAgain

Time (mean ± σ):      40.7 ms ±   0.8 ms    [User: 39.2 ms, System: 1.4 ms]
Range (min … max):    39.7 ms …  45.7 ms    70 runs

HEAD is now at 996b784 Fixes for expression

Time (mean ± σ):      40.6 ms ±   0.4 ms    [User: 38.6 ms, System: 2.0 ms]
Range (min … max):    39.8 ms …  42.4 ms    71 runs

HEAD is now at c81b62f chore: generate latest changes

Time (mean ± σ):      40.7 ms ±   1.6 ms    [User: 39.0 ms, System: 1.6 ms]
Range (min … max):    40.1 ms …  53.4 ms    71 runs

HEAD is now at cbf8207 Extra highlights for scala 3 constructs (#120)

Time (mean ± σ):      41.4 ms ±   6.9 ms    [User: 38.6 ms, System: 2.1 ms]
Range (min … max):    39.7 ms …  98.9 ms    71 runs

HEAD is now at 24daf6e Merge pull request #118 from eed3si9n/wip/spec-correct

Time (mean ± σ):      41.1 ms ±   4.1 ms    [User: 38.8 ms, System: 1.8 ms]
Range (min … max):    39.8 ms …  74.8 ms    71 runs

HEAD is now at 57a0e42 dotty/compiler/: expected 72, but got 71.99 instead

Time (mean ± σ):      41.1 ms ±   2.4 ms    [User: 39.1 ms, System: 1.7 ms]
Range (min … max):    39.7 ms …  58.3 ms    71 runs

HEAD is now at 4bb8fab Don't use stable_identifier for imports

Time (mean ± σ):      41.0 ms ±   2.8 ms    [User: 38.8 ms, System: 2.0 ms]
Range (min … max):    39.7 ms …  63.3 ms    69 runs
@keynmol keynmol changed the title Benchmark grammar on CI Benchmark parser on CI Jan 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant