Editing Markdown pipe tables

[bpj]

(Apologies in advance if this is a duplicate. I could have sworn that I had raised this before but I can't find it now.)

I'm using this plugin to edit Markdown pipe tables (supported by at least GFM and Pandoc) after opening them in a NrrwRgn window. As the name implies they use | as a delimiter, so it mostly works out of the box, but there are some problems:

After having edited a pipe table it usually looks somewhat like this:

|foo   |bar  |baz
|:---  |---: |---:
|quux  |1    |34
|blort |56   |789

or (even worse) like this:

|foo|bar|baz
|:---|---:|---:
|quux|1|34
|blort|56|789

but I would like it to look like this:

| foo   | bar  | baz
|:------|-----:|----:
| quux  | 1    | 34
| blort | 56   | 789

or even better like this:

| foo   | bar | baz
|:------|----:|----:
| quux  |   1 |  34
| blort |  56 | 789

TL;DR

It boils down to this:

In my ~/.vim/after/ftplugin/markdown.vim I have this which does fix the formatting of pipe tables to my satisfaction:

" Use tabular.vim to align the pipes
" properly ignoring backslash escaped pipes
com! -range PtabReform <line1>,<line2>Tabularize /\v%(\\%(\\\\)*)@<!\|/l1

" Fix up the header separator line,
" including making sure that the
" alignment marking colons are next
" to the pipes
fun! s:PtabSep ()
    .s/\v(\s+)(\:)\ze\-/\2\1/ge
    .s/\v\-\zs(\:)(\s+)/\2\1/ge
    .s/\v[^|:]/-/ge
endfun

com! PtabSep call <sid>PtabSep()

While this works I would like to have csv.vim be "smart" about these things. So I guess that what I'm wondering is basically:

How can I write a column recognition pattern which is the ethical equivalent of this Perl regex which I had in my reformating script which I abandoned when I found that :Tabularize can handle the realignment:

qr{
    # A *leading* pipe character
    (?<= \| )
    # followed by zero or more chars
    # which are not pipes or backslashes
    # (normal characters)
    ( [^|\\]*
    # followed by zero or more repetitions
    # of a backslash escaped character
    # followed by zero or more normal chars
    (?: \\. [^|\\]* )* )
    # optionally followed by a pipe at
    # the end of line
    (?= \| \s* $ )?
}mx

Note that I used to strip leading/trailing whitespace from the captured part and then reinsert it after calculating column widths. I don't know if it matters.

Detailed description

1. At least Pandoc supports backslash escaped characters, including backslash escaped pipes inside cell values.

   At the basic level I'm having problems defining a b:col expression to match this. I have tried with

   \%(\%(\%(\s*|\%([^|\\]\|\\.\)*|\s*\)\%(|\|$\)\)\|\%([^|\\]*\%(|\|$\)\)\)
   

   but it doesn't seem to work.

   Ideally I would want a b:escape=CHAR variable and the b:col pattern which eludes me being correctly created if the escape is different from the delimiter.

2. Pipe tables (at least mine!) are usually formatted with whitespace around the pipe for readability, which the Markdown parsers ignore, i.e. it's usually

   | foo   | bar | baz
   |:------|----:|----:
   | quux  |   1 |  34
   | blort |  56 | 789

   rather than

   |foo|bar|baz
   |:---|---:|---:
   |quux|1|34
   |blort|56|789

   :ArrangeColumn will currently only insert leading or trailing whitespace; I would like to be able to have a single space inserted at the "other end". Something like an g:insert_ws option which does what I do now along the lines of :%s/|/ | /g but with the pattern adjusted for ignoring existing whitespace and escapes, something like

   :%s/\x20\?\%(\\\%(\\\\\)*)@<!|\x20\?/ | /g

3. In a pipe table the pipes denote the start of each cell so there is always a leading pipe in the inner margin. This sometimes causes problems since the plugin will insert whitespace between the margin and that pipe. Also a trailing pipe in the outer margin is legal, although I usually don't use it. Sure I can do :%s/^\s* and :%s/\s*$ when I'm done, but I keep forgetting. Perhaps something like a pair of configurations b:ignore_leading_delimiter and b:ignore_trailing_delimiter which would define the column detecting pattern to ignore anything matching ^\s*D and D\s*$, where D is the delimiter might be made to work?

4. A "headerless" table is marked up as a top row with just a pipe in the inner margin, which confuses this plugin.

   |
   |:------|----:|----:
   | quux  |   1 |  34
   | blort |  56 | 789

   I don't know if it would help to be able to configure :CSVInit to ignore a given number of lines at the top.

5. Pipe tables support per-column alignments, marked with colons in the head separator row. Let's just say that it is annoying to have to choose when there is both text and number data in a table! (There is also :---: for center aligment, although at least I would be fine with it being left-aligned for display.) Ideally I would like to have something like a :let b:alignments='l,c,r', with the last alignment being repeated if there are more columns than specified alignments.

I'm not asking for something as ambitious as a "pipe table mode", just some config to make escape/delimiter/whitespace handling a little more flexible, which hopefully would be more generally useful. I really would like to contribute this myself but the truth is that I'm not sure if these things could be handled by altering how the plugin builds its column matching pattern. I have tried and failed to build a pattern which suits my need. If such configuration were possible one could write one's own function/command to apply such configuration as and when one wants to.

I fully understand if this is out of scope, so see it as a suggestion rather than as a request (as in no big deal because i can do without it).

[bpj]

WRT point 5 above I have "discovered" b:csv_arrange_align so that is "fixed".