WeBWorK 2.19 support (Tacoma 2024 project)

[Alex-Jordan]

This is a large PR, just in time for everyone to be going on vacation :)

I'm doing my best to group the contributions from @drgrice1. @drdrew42, and myself from Tacoma. According to my testing, with these changes:

1. Nothing significantly changes (using the minimal webwork example and the sample-chapter) when you use a WeBWorK 2.17 or 2.18 server, like the AIM webwork-ptx server or the Runestone book support server. It would be good if we work out a way to test that Runestone books aren't broken if this is merged and makes its way to the CLI.
2. Things work (and work better) when there is a WeBWorK 2.19 server involved. The AIM webwork-dev server is using 2.19.

Once using a 2.19 server, there are new features that I have tested and documented using the minimal webwork example and the sample-chapter.

• You can use local .pg files in the external folder.
• You can do the static processing of PG using a local instance of the pg repository. For me in testing today, this took close to 1/4 the time of using the AIM webwork-dev server. You do need to install that pg repo is all.
• You can do the static processing of PG using an instance of the standalone renderer that is also located on webwork-dev. For me in testing today, this took close to 80% of the time of using the AIM webwork-dev server.

Possibly, we need a schedule for pulling the trigger on this. Here's a draft.

1. Rob reviews.
2. Alex amends according to Rob's review
3. Work with @bnmnetp to check that RS books are not adversely affected.
4. Rob merges.
5. Work with @bnmnetp to also check that RS books are not adversely affected if the RS support server moves to version 2.19.
6. Alex upgrades both the RS support server and AIM's webwork-ptx to version 2.19.

[drgrice1]

Thanks for putting this together for us.

[rbeezer]

I've checked all of this against existing behavior, and will have some comments in a minute.

But, I don't see any reason to not start testing behavior on Runestone now, step (3) of above with @bnmnetp.

[rbeezer]

Thanks, @Alex-Jordan for putting all of this together. Some comments:

• Are you sure you want to load LaTeX macros into an attribute (@data-macros)? Could they not be accessed from an element? Looks mildly dangerous to me.
• I think "choice" type problems are losing their \item in the conversion to LaTeX lists. Sample Chapter, Checkpoint 1.6.1 about sqrt(2) being rational or not, is a good first example.
• Using the makefile to generate problem sets seems to land them someplace much deeper than expected in the output. Might just be a problem with the makefile, I haven't investigated too deep.

Not much, given the quantity of code. Good job. I'm going to familiarize myself with the new features now, so may have more to say. But wanted to get this going in the right direction as quickly as possible.

[rbeezer]

Using the makefile to generate problem sets seems to land them someplace much deeper

OK, I see rendering-data in a file now, I was confused by the diff I was looking at. Scratch that comment.

[rbeezer]

And with AIM's 2.19 server, the \item in the LaTeX conversion are back, so this may just be an issue for backward compatibility.

[rbeezer]

• Built with AIM dev server in publication file. Problems hang with "Loading" when I click Activate. Firefox and Chrome.
• Guide has two sections "Using a local copy of PG", which need to be edited/merged?

[rbeezer]

@static-processing="local"

Can't locate Mojo/Base.pm in @INC (you may need to install the Mojo::Base module) (@INC entries checked: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.38.2 /usr/local/share/perl/5.38.2 /usr/lib/x86_64-linux-gnu/perl5/5.38 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl-base /usr/lib/x86_64-linux-gnu/perl/5.38 /usr/share/perl/5.38 /usr/local/lib/site_perl) at /home/rob/mathbook/mathbook/script/webwork/pg-ptx.pl line 3.
BEGIN failed--compilation aborted at /home/rob/mathbook/mathbook/script/webwork/pg-ptx.pl line 3.

Ideally, an Ubuntu package would provide what I need?

https://packages.ubuntu.com/oracular/libmojolicious-perl

[drgrice1]

Yes, the libmojolicious-perl package provides what is needed. That is the primary down side of the local approach. It does require that the necessary Perl modules be installed.

[rbeezer]

Thanks, @drgrice1! I needed the following Ubuntu packages to get through the sample chapter.

libmojolicious-perl
liblocale-maketext-lexicon-perl
libgd-perl
libdbi-perl
libuuid-tiny-perl
libclass-tiny-perl

[rbeezer]

Built representations throug to the end. Some non-fatal errors. About 6 instances, problem numbers 62-68 (or so), and maybe the very last one. Can't tell what I might be missing.

PTX:INFO    : sending webwork-62 to socket to save in /home/rob/mathbook/mathbook/examples/webwork/sample-chapter/generated/webwork/webwork-representations.xml: origin is 'generated'
errors:
ERRORS from evaluating PG file:
Undefined subroutine &main::latexImagePreamble called at line 7 of 
warnings:
Can't locate macro file |WWSC.pl| via path: |.|,<br/> |[PG]/macros/|,<br/> |[PG]/macros//answers|,<br/> |[PG]/macros//capa|,<br/> |[PG]/macros//contexts|,<br/> |[PG]/macros//core|,<br/> |[PG]/macros//deprecated|,<br/> |[PG]/macros//graph|,<br/> |[PG]/macros//math|,<br/> |[PG]/macros//misc|,<br/> |[PG]/macros//parsers|,<br/> |[PG]/macros//ui|
PTX:ERROR: WeBWorK problem generated/webwork/pg/Integrating_WeBWorK_into_Textbooks/1_8_1-A_static_lateximage_graph.pg with seed 62 is either empty or failed to compile  
  Use -a to halt with full PG and returned content

[Alex-Jordan]

Thanks Rob! Some quick responses and I may do more this evening.

Are you sure you want to load LaTeX macros into an attribute (@data-macros)? Could they not be accessed from an element? Looks mildly dangerous to me.

What's going on here is the PTX HTML page has a div with id latex-macros. The contents of that div will be the LaTeX macros, but wrapped in \(...\). Now, later the webwork js needs to get the latex macros to write them into the srcdoc for an iframe where the WW problem will render. So how will it get them? I have it accessing them from where they are stored redundantly as a data attribute on the div#latex-macros (without the \(...\)). It's been a while, but I think I felt uneasy with having it access them from the content of the div#latex-macros. Particularly because the \(...\) would already be present. Possibly it could change to access them from the content and remove the redundant data attribute. If there is a future need to add macros at this stage, either I'd have to peel off the \(...\) or revert to how it is now.

I think "choice" type problems are losing their \item in the conversion to LaTeX lists. Sample Chapter, Checkpoint 1.6.1 about sqrt(2) being rational or not, is a good first example. And with AIM's 2.19 server, the \item in the LaTeX conversion are back, so this may just be an issue for backward compatibility.

OK, I will follow up with this.

Using the makefile to generate problem sets seems to land them someplace much deeper than expected in the output. Might just be a problem with the makefile, I haven't investigated too deep.

I neglected to test with the makefile but I'll do that to make sure things are still working. I was testing by running pretext/pretext/pretext directly. But actually this sounds like an expected change (that I forgot to mention in all the mix of things). Prior to this PR, when you make an archive of PG files, that gets made wherever you happen to be. Or possibly you can specify a -d or a -o. But now it all gets placed in an appropriate subfolder of the generated folder. I think this is a good change.

Built with AIM dev server in publication file. Problems hang with "Loading" when I click Activate. Firefox and Chrome.

Are you viewing with file://? Same thing happened to me if I didn't view through an actual web server.

Guide has two sections "Using a local copy of PG", which need to be edited/merged?

Sounds like an oversight. Will follow up.

Ideally, an Ubuntu package would provide what I need?

I'll add to the documentation as best I can about what packages are needed. Or maybe a generic "you may need to install some perl packages". With Ubuntu packages, using cpanm, and more variants, I'm not sure about getting too specific.

Undefined subroutine &main::latexImagePreamble

This indicates a host course doesn't have the access it needs to the .pl file for WWSC. I'll look into it.

[drgrice1]

Undefined subroutine &main::latexImagePreamble

This indicates a host course doesn't have the access it needs to the .pl file for WWSC. I'll look into it.

I think that this was when @rbeezer was testing the local approach based on the output line "sending webwork-62 to socket". The local approach adds the macros directory in the generated directory to the macro path. So the macro should be available. Although, I also don't see that path in the output that @rbeezer showed.

[Alex-Jordan]

I think "choice" type problems are losing their \item

Looking over the diff, I think this is from -assembly where I replaced var[@form] with ul, ol, dl as appropriate. I think if I bring back support for var[@form] as equivalent to ul[@form], that will address the backwards compatibility.

[Alex-Jordan]

I think "choice" type problems are losing their \item

OK, this one is addressed. I added a commit (rather than force push) but it can be squashed later as it indicates.

[Alex-Jordan]

Guide has two sections "Using a local copy of PG", which need to be edited/merged?

Copy-paste oversight. The second one should have had its title changed to be about using a renderer.

[Alex-Jordan]

@drgrice1 Do you have a recommendation about what I should put in the documentation about which perl version (or later) is required for local PG processing?

[drgrice1]

I am not exactly sure how old of a Perl version will work for local PG processing. I know that Perl 5.16 or newer is certainly needed. I suspect that versions that old will have problems though. I think that 5.26 is probably the oldest that we should try to support, and so I guess put that in the documentation. Although you would need 5.32 or newer for a version that I can truly guarantee everything will work for.