fuzzer: add special-case for multi-line EOF TokenError #1991
Conversation
jayaddison
changed the title
fuzz.py
Outdated
| ): | ||
| print('Warning: skipped fuzzer-generated input that contained a backslash') | ||
| print('See https://github.com/psf/black/issues/1012 for details') | ||
| print('To replay this fuzzer run, find the PYTHONHASHSEED environment value') |
There was a problem hiding this comment.
@Zac-HD Could you confirm whether this is reasonable advice?
Ah. If we're not just waiting out the end-of-life for Python 3.7, I think there's an actual bug which should be fixed. Concretely for this PR though, I'd just remove the version-check and change the comment to explain that we're suppressing a known bug and will fix it later. |
Co-authored-by: Zac Hatfield-Dodds <[email protected]>
…recent Python versions than 3.7
|
Is there an existing upstream issue that we can link to here, or perhaps we should create one? (my thinking is that we should add a note/reference upstream to remind us to clean this workaround up once the root cause is resolved) |
|
I think the bug is in Black - Hypothesmith is generating source code which is valid according to the compile() builtin, and if Black can't handle it the bug is in Black's tokenisation, not upstream. If you think it is upstream, where is it and how should it be fixed? |
|
Ok, yep. It does follow logically that this is most likely a bug in |
fuzz.py
Outdated
| import hypothesmith | ||
| from hypothesis import HealthCheck, given, settings, strategies as st | ||
|
|
||
| import black | ||
| from blib2to3.pgen2.tokenize import ( # NB: This intends to (and currently does) import the black-vendored blib2to3 |
There was a problem hiding this comment.
I added this comment based on a memory of reading some code that variously imported either the blib2to3 code or Python's built-in lib2to3 code. But after checking it again, I realize they use different module names, so I think this comment is redundant and can be removed.
This reverts commit 357b7cc.
|
This is awesome work all. @jayaddison - Appreciate all your efforts, I know I haven't been around much. But appreciate the deep debugging and pulling these PRs together! |
|
No worries, thanks @cooperlees! |
Goal: reduces test failure false-positives relating to #1012.
cc @Zac-HD