Redirected Output

[stevesims]

Currently RST10 and RST18 output only to the VDP

It would be nice to be able to redirect output to a file handle, and potentially to multiple handles simultaneously. This would allow for an Acorn-style *SPOOL to be implemented - in short, it makes it easy to print out to a file.

This is most likely strongly related to #80 - the existing VDP connection could/should be given a file handle, same as other connections.

[stevesims]

On RISC OS you could set the redirection as part of a command - redirection info would need to be at the end of a command line and be enclosed in squiggly brackets, for instance:

command { > filepath }
command { >> filepath }
command { < filepath }

The first of these would send output to a file, second would append to that file, and the third sets the input stream as a particular file. These essentially set the input/output streams to use whilst the command is being executed. The default input stream is the keyboard, and the default output stream the VDU.

(The squiggly's are needed because plain > or < could be part of a variable or raw character number.)

the filepath could optionally point to a system device. Available options on RISC OS are as follows:
kbd: keyboard, input only, reading a line at a time (allowing for line editing)
rawkbd: keyboard, input only, reading a character at a time
serial: the serial port, for input and output
parallel: the parallel port, for input and output
null: null device, for input and output
vdu: screen output, using GSRead format
rawvdu: screen output, sent direct to VDU
printer: and netprinter: output only to currently configured printer or network printer

Such device paths could be used for targets in commands, such as copy or cat

RISC OS has no concept of an error stream, so error output is not differentiated from regular output.

PANOS on the other hand did have an error: stream. Usually it was associated with the screen. All commands on PANOS would have a standard argument of -ERRor (presumably either err or error) to set an alternative stream for error output. There were also standard arguments for setting input and output streams.

Unix shells tend to support redirection using the same arrow style operators as RISC OS. Changing the output of stderr is done by prefixing the arrow(s) with a 2, so for instance 2>errors.txt sets stderr to output to errors.txt. Prefixing with a 1 redirects stdout, so 1>file.txt and >file.txt are essentially equivalent.

[stevesims]

Redirected output presents us with an interesting backwards compatibility problem. And to a lesser extent a forwards compatibility problem too.

The primary problem with the Agon platform as it has been implemented so far with redirecting the output is that our default "output stream" is RST10/RST18, which is the channel to the VDP. When a program only produces text output to screen this is fine, however if a program needs to run any command to query the VDP then if we have redirected RST10/RST18 then those commands would get sent to the redirected output, and not make it to the VDP.

In MOS we don't have a separate API for "output text". Similarly we don't have any specific MOS APIs to "query VDP". We have no way of telling the difference between text output and a command intended for the VDP. (If our VDU command set "followed the rules" of fixed length commands as Acorn had designed then that might have been possible, but nope.)

Non-interactive programs, for the most part, don't need to send VDP commands that expect a response back from the VDP. We have only a relatively small number of commands that send a response; all such responses are automatically handled by MOS, filling in sysvar values. Most sysvars get their values automatically updated by data pushed from the VDP, rather than needing a command to pull them. There are very few sysvars that need to be "pulled" - specifically cursorX, cursorY, scrchar, scrpixel and rtc. Of these only rtc has a MOS API that would trigger a fetch of new RTC data. The other values are not very likely to be required for a non-interactive program. (Arguably we could attempt to trap those few VDU commands and set the VDP appropriate protocol flag and spoof a response.)

The only specific MOS API we have that is relevant here is mos_getrtc, and that could be changed to ensure the VDP command it sends goes over the UART to the VDP. (It should be noted however that since that API call returns a pre-formatted string, it may not be the most desirable way to get RTC information for some programs.)

Ideally MOS should offer APIs to specifically request for the sysvars to be updated. Unfortunately earlier versions of MOS will crash when a program attempts to use an undefined API call. Hence the interesting backwards/forwards compatibility problem.

To be clear, in reality the number of non-interactive programs that would be likely to need to send commands to the VDP to query for results is small. Therefore this is not actually a big problem. We could add redirection that simply captures RST10/RST18 for the duration of a program run and this will work for the majority of such programs.

[stevesims]

As noted above, the primary issue with redirected output is that it is essentially impossible to tell the difference between output that's commands to control the VDP, or query the VDP for data, and genuine output. Generally it is VDU 23 commands that are used to control/query the VDP - and those we would want/need to intercept and prevent from appearing in redirected output. Unfortunately the Agon's VDU 23 uses variable length commands this is not a practical proposition to perform from within MOS.

A potential solution to this would be to use a "loopback" mechanism - have the VDP send data packets back to MOS with output when redirected output is enabled

This would allow the VDP to filter out commands, and echo back all other bytes

[stevesims]

on RISC OS, whilst vdu: and rawvdu: are valid paths to reference the VDU, they do not work as a redirection target (on RISC OS 3 at least). performing a . { > vdu: } does nothing - it seems to be equivalent to redirecting to null:. Using them as a target for copy however does work, so copy file vdu: is essentially equivalent to type file

Whilst you can specify multiple redirection targets in RISC OS, only the last target will be used - although files will get created for all output targets.

RISC OS redirection directives require spaces around directive elements. so {> test.txt } or {>test.txt} are invalid - the redirection must be { > test.txt }. Conversely the opening { does not need a space before it, so .{ > dir } works.

[stevesims]

attempts to implement spool in #117 , which effectively is redirection without preventing the VDP output, are unfortunately at this time not successful and proving problematic.

the approach taken was the "loopback" idea mentioned above. on receipt of an "echo" VDP protocol packet MOS needs to action saving the contents of that packet to disc. experiments show problems when more than one packet is received in rapid succession - typically a hard-crash with MOS rebooting. it is not clear what exactly causes this crash. techniques to try to ensure that write operations don't overlap do not seem to help. a spool of very short "echo" statements can succeed. without the forced delay between echo packets, longer "echo" strings cause crashes. attempting to perform a *dir results in a stalled machine - presumably because of the overlap between reading directory entries and writing out the "echo" packet, which is a result of the write out starting from an interrupt whilst the directory read is still in progress

to try to ensure that read and write operations cannot overlap will require some significant effort - ideas for how to resolve this involve things like "perform write operations in a separate task", which given that MOS currently doesn't support "tasks" would be hard