Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert several perl scripts to python #511

Merged
merged 17 commits into from
Dec 29, 2024
Merged

Convert several perl scripts to python #511

merged 17 commits into from
Dec 29, 2024

Conversation

charlesstaats
Copy link
Contributor

Convert several perl scripts to python.

Python is now an installation dependency in any case, and the current asy developers are generally more familiar with python than with perl. Thus, converting the perl code to python should make it easier to maintain and improve these files.

Because I have noticed improvements being made to the perl scripts, I want to go ahead and get these merged even though I am still working on the most complex perl script, runtime.pl.

The translations are based on AI-provided translations but thoroughly checked by me (with numerous modifications in the process). They could of course still contain bugs, but I am confident they are not worse than what I would produce if I tried to do this without the aid of AI.

For testing, I added a log statement to the original perl script to figure out exactly how it was being used during make, saved the results, and compared them to the results of the python generation. In all cases, the generated files differed by exactly one character, which was the name of the generating file (something.pl changed to something.py) in the comment block.

Note: I have only tested the changes using the make utility. I do not know how to compile with cmake, although I have made an effort to change the cmake files as well.

Testing complication: At least on my machine, if I run a python file using perl, and it sees the python shebang line, it will call the python interpreter rather than interpreting the file as if it were perl. This could make it easy to miss if we're accidentally running python files with perl.

The comments and python code were originally generated by AI based on
the original asy-list.pl code; however, both were subsequently checked
and modified by a human to improve both Python coding style and accuracy
to the original script.

The change was tested by comparing the output of `make asy-keywords.el`
before and after the change. The resulting files are identical except
for the comment naming the generating script, which was originally
asy-list.pl but is now asy-list.py. Thus, the two output files differ by
a single character.
Tested: I used a logging statement in the original `findsym.pl` to check
that all of the files written by `findsym.pl` followed the pattern
`*symbols.h`. I then checked that after the change, the resulting files
differed from the originals by a single character (since the comment now
cites `findsym.py` rather than `findsym.pl` as the source). The sole
exception was `opsymbols.h`, which had no changes at all since it is
generated by a different script.
@jamievlin
Copy link
Member

Can you ensure the CI passes before merging this in?

Also, could you check that the python scriptd and the old perl files produce the same output?

Another thing - the shebang lines should be "python3", not "python"

@charlesstaats
Copy link
Contributor Author

Here's a list of all the diffs of potentially generated files, comparing master versus the current branch. I should note that in addition to the *.pl->*.py changes I noted in the initial comment, there is also a change based on the asy version.

$ for file in old/*; do echo "diffing ${file}:"; diff "${file}" "${file##*/}"; done
diffing old/Delaunay.cc:
diffing old/Delaunay.h:
diffing old/EXRFiles.cc:
diffing old/EXRFiles.h:
diffing old/GLTextures.cc:
diffing old/GLTextures.h:
diffing old/abs3doutfile.h:
diffing old/absyn.cc:
diffing old/absyn.h:
diffing old/access.cc:
diffing old/access.h:
diffing old/align.h:
diffing old/allsymbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/angle.h:
diffing old/application.cc:
diffing old/application.h:
diffing old/array.cc:
diffing old/array.h:
diffing old/arrayop.h:
diffing old/asy-keywords.el:
2c2
< ;; This file is automatically generated by asy-list.pl.
---
> ;; This file is automatically generated by asy-list.py.
5c5
< (defvar asy-keywords-version "2.96-83")
---
> (defvar asy-keywords-version "2.96-97")
diffing old/asyparser.cc:
diffing old/asyparser.h:
diffing old/asyprocess.cc:
653c653
<     // keywords.pl looks for ADDCOMMAND to identify special commands in the
---
>     // keywords.py looks for ADDCOMMAND to identify special commands in the
diffing old/asyprocess.h:
diffing old/bbox.h:
diffing old/bbox3.h:
diffing old/beziercurve.cc:
diffing old/beziercurve.h:
diffing old/bezierpatch.cc:
diffing old/bezierpatch.h:
diffing old/builtin.cc:
diffing old/builtin.h:
diffing old/builtin.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/callable.cc:
diffing old/callable.h:
diffing old/camp.tab.cc:
diffing old/camp.tab.h:
diffing old/camperror.cc:
diffing old/camperror.h:
diffing old/castop.h:
diffing old/coder.cc:
diffing old/coder.h:
diffing old/coenv.cc:
diffing old/coenv.h:
diffing old/common.h:
diffing old/config.h:
diffing old/constructor.cc:
diffing old/dec.cc:
diffing old/dec.h:
diffing old/drawclipbegin.h:
diffing old/drawclipend.h:
diffing old/drawelement.h:
diffing old/drawfill.cc:
diffing old/drawfill.h:
diffing old/drawgrestore.h:
diffing old/drawgroup.h:
diffing old/drawgsave.h:
diffing old/drawimage.h:
diffing old/drawlabel.cc:
diffing old/drawlabel.h:
diffing old/drawlayer.h:
diffing old/drawpath.cc:
diffing old/drawpath.h:
diffing old/drawpath3.cc:
diffing old/drawpath3.h:
diffing old/drawsurface.cc:
diffing old/drawsurface.h:
diffing old/drawverbatim.h:
diffing old/entry.cc:
diffing old/entry.h:
diffing old/env.cc:
diffing old/env.h:
diffing old/envcompleter.cc:
diffing old/envcompleter.h:
diffing old/errormsg.cc:
diffing old/errormsg.h:
diffing old/exithandlers.cc:
diffing old/exithandlers.h:
diffing old/exp.cc:
diffing old/exp.h:
diffing old/fftw++.cc:
diffing old/fftw++.h:
diffing old/fftw++asy.cc:
diffing old/fileio.cc:
diffing old/fileio.h:
diffing old/flatguide.cc:
diffing old/flatguide.h:
diffing old/fpu.h:
diffing old/frame.h:
diffing old/fundec.cc:
diffing old/fundec.h:
diffing old/genv.cc:
diffing old/genv.h:
diffing old/glrender.cc:
diffing old/glrender.h:
diffing old/gsl.cc:
diffing old/gsl.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/guide.cc:
diffing old/guide.h:
diffing old/guideflags.h:
diffing old/impdatum.cc:
diffing old/inst.h:
diffing old/interact.cc:
diffing old/interact.h:
diffing old/item.h:
diffing old/jsfile.cc:
diffing old/jsfile.h:
diffing old/keywords.cc:
diffing old/keywords.h:
2c2
<  * This file is automatically generated by keywords.pl.
---
>  * This file is automatically generated by keywords.py.
diffing old/knot.cc:
diffing old/knot.h:
diffing old/lex.yy.cc:
889c889
<  * The script opsymbols.pl reads this file scanning for rules using DEFSYMBOL
---
>  * The script opsymbols.py reads this file scanning for rules using DEFSYMBOL
diffing old/lexical.h:
diffing old/locate.cc:
diffing old/locate.h:
diffing old/lspserv.cc:
diffing old/lspserv.h:
diffing old/main.cc:
diffing old/makeUnique.h:
diffing old/material.h:
diffing old/mathop.h:
diffing old/memory.cc:
diffing old/memory.h:
diffing old/mod.h:
diffing old/modifier.h:
diffing old/name.cc:
diffing old/name.h:
diffing old/newexp.cc:
diffing old/newexp.h:
diffing old/opcodes.h:
diffing old/opsymbols.h:
2c2
<  * This file is automatically generated by opsymbols.pl
---
>  * This file is automatically generated by opsymbols.py
diffing old/pair.h:
diffing old/parallel.cc:
diffing old/parallel.h:
diffing old/path.cc:
diffing old/path.h:
diffing old/path3.cc:
diffing old/path3.h:
diffing old/pen.cc:
diffing old/pen.h:
diffing old/picture.cc:
diffing old/picture.h:
diffing old/pipestream.cc:
diffing old/pipestream.h:
diffing old/policy.h:
diffing old/prcfile.h:
diffing old/predicates.cc:
diffing old/predicates.h:
diffing old/primitives.h:
diffing old/profiler.h:
diffing old/program.cc:
diffing old/program.h:
diffing old/psfile.cc:
diffing old/psfile.h:
diffing old/record.cc:
diffing old/record.h:
diffing old/refaccess.cc:
diffing old/refaccess.h:
diffing old/revision.cc:
1c1
< const char *REVISION="2.96-83";
---
> const char *REVISION="2.96-97";
diffing old/rounding.h:
diffing old/runarray.cc:
diffing old/runarray.h:
diffing old/runarray.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runbacktrace.cc:
diffing old/runbacktrace.h:
diffing old/runbacktrace.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runfile.cc:
diffing old/runfile.h:
diffing old/runfile.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runhistory.cc:
diffing old/runhistory.h:
diffing old/runhistory.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runlabel.cc:
diffing old/runlabel.h:
diffing old/runlabel.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runmath.cc:
diffing old/runmath.h:
diffing old/runmath.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runpair.cc:
diffing old/runpair.h:
diffing old/runpair.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runpath.cc:
diffing old/runpath.h:
diffing old/runpath.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runpath3d.cc:
diffing old/runpath3d.h:
diffing old/runpath3d.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runpicture.cc:
diffing old/runpicture.h:
diffing old/runpicture.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runstring.cc:
diffing old/runstring.h:
diffing old/runstring.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runsystem.cc:
diffing old/runsystem.h:
diffing old/runsystem.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runtime.cc:
diffing old/runtime.h:
diffing old/runtime.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/runtriple.cc:
diffing old/runtriple.h:
diffing old/runtriple.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/seconds.h:
diffing old/settings.cc:
diffing old/settings.h:
diffing old/shaders.cc:
diffing old/shaders.h:
diffing old/simpson.cc:
diffing old/stack.cc:
diffing old/stack.h:
diffing old/statistics.h:
diffing old/stm.cc:
diffing old/stm.h:
diffing old/symbol.cc:
diffing old/symbol.h:
diffing old/symbolmaps.cc:
diffing old/symbolmaps.h:
diffing old/table.h:
diffing old/texfile.cc:
diffing old/texfile.h:
diffing old/tr.cc:
diffing old/tr.h:
diffing old/transform.h:
diffing old/triple.h:
diffing old/types.cc:
diffing old/types.h:
diffing old/types.symbols.h:
2c2
<  * This file is automatically generated by findsym.pl
---
>  * This file is automatically generated by findsym.py
diffing old/util.cc:
diffing old/util.h:
diffing old/v3dfile.cc:
diffing old/v3dfile.h:
diffing old/v3dheadertypes.h:
diffing old/v3dtypes.h:
diffing old/varinit.cc:
diffing old/varinit.h:
diffing old/virtualfieldaccess.cc:
diffing old/virtualfieldaccess.h:
diffing old/vm.h:
diffing old/win32helpers.cc:
diffing old/win32helpers.h:
diffing old/win32pipestream.cc:
diffing old/win32pipestream.h:
diffing old/win32xdr.cc:
diffing old/win32xdr.h:
diffing old/xdrcommon.h:
diffing old/xstream.cc:
diffing old/xstream.h:

@charlesstaats charlesstaats merged commit bb5b878 into master Dec 29, 2024
17 checks passed
@jamievlin
Copy link
Member

Also, just wondering, but is there a reason to use ascii encoding instead of UTF8? At least from my experience, I always use UTF-8 unless if there's a reason to use ASCII

@charlesstaats
Copy link
Contributor Author

I was mainly concerned with fixing the lint errors. My reasoning, for the 30 seconds or so I spent thinking about this, was that if we have non-ascii characters in our C++ code (which is what these scripts are reading and writing, with the exception of asy-keywords.el), there's probably something wrong. But if the ascii encoding had created a problem or diff that could be fixed by moving to utf8, and an investigation found this was not an issue with the C++ files, I would not have had a problem moving to utf8. Additionally, if you have a convincing reason to move to utf8, or even just feel strongly about it, and want to change it, you will hear no objections from me. (That goes for a lot of other changes to the python files as well; a large part of the point of migrating to python was to enable changes and improvements. Is ascii the right encoding in the python code? We can discuss that. Is ":unix:clfr" the right binmode layer in the perl code? No idea, I can't even google it properly because search engines ignore colons.)

Note that for scripts that read and write tex code or asy code (which can include tex code), I agree that utf8 is almost certainly preferred.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants