Feedback: Output UX

[polarathene]

q is a great DNS utility although the UX has a little bit of friction. I'm not sure how much of these you can address easily while keeping the same size advantage over alternatives like doggo.

Column Alignment

Readability could be improved if the record values were column aligned? The TTL and record types otherwise make that less pleasant to scan through.

You can kind of workaround it by requesting a specific record type, or using --short, but for multiple records where alignment is useful, it can be helpful to have the context of the record type.

Here's how it looks from doggo:

Column Headers

Not too important, and definitely should be optional (opt-in is fine), this seems to compliment the column alignment feature as shown in the doggo example above.

Likewise, I don't think you can filter columns (eg: If I'm interested in the equivalent of --short but additionally with the record type).

doggo doesn't appear to support disabling the headers line, or columns displayed. Just a feature I've seen in some other CLI tools when presenting results.

---

Default opt-out flags (+[no])

I wanted to try disable the default --pretty-ttls feature early on, and saw the help mention of dig notation, but that wasn't immediately obvious to me to figure out and I failed a few times until landing on this issue comment to realize the option replaces -- with + or +no.

--pretty-ttls false or --no-pretty-ttls would seem a bit more natural? This particular setting could be a preference as a default, but there is no config or generic ENV support, I suppose you could use a shell alias 🤷‍♂️

Those are nice to haves, but I think just having an example on your README with the +no syntax would be sufficient, --pretty-ttls is probably a good example.

Pretty TTL

In my case for 60s, the pretty TTL renders as 1m0s, and the README example 1 day is 24h0m0s. The 0 units could maybe be truncated as they're added noise?

---

--format json

This differs from doggo which actually outputs a JSON document array of records.

Your JSON output is technically JSONL (JSON Lines), which can be useful too. Just thought I'd point out that difference from what I was expecting to see.

Tools like yq could convert the YAML format into JSON easily enough. While jq can convert JSONL to a pretty printed JSON array with q example.test NS MX TXT A --format json | jq --slurp '.':

Q JSONL output

{"Server":"127.0.0.11:53","QueryTime":2000000,"Answers":[{"Hdr":{"Name":"example.test.","Rrtype":15,"Class":1,"Ttl":60,"Rdlength":9},"Preference":10,"Mx":"mail.example.test."}],"ID":34374,"Truncated":false}
{"Server":"127.0.0.11:53","QueryTime":2000000,"Answers":[{"Hdr":{"Name":"example.test.","Rrtype":16,"Class":1,"Ttl":60,"Rdlength":15},"Txt":["v=spf1 mx -all"]}],"ID":3010,"Truncated":false}
{"Server":"127.0.0.11:53","QueryTime":2000000,"Answers":[{"Hdr":{"Name":"example.test.","Rrtype":1,"Class":1,"Ttl":60,"Rdlength":4},"A":"172.16.42.11"}],"ID":57132,"Truncated":false}
{"Server":"127.0.0.11:53","QueryTime":2000000,"Answers":[{"Hdr":{"Name":"example.test.","Rrtype":2,"Class":1,"Ttl":60,"Rdlength":6},"Ns":"ns1.example.test."}],"ID":22482,"Truncated":false}

Q JSONL => JSON output (via jq)

[
  {
    "Server": "127.0.0.11:53",
    "QueryTime": 2000000,
    "Answers": [
      {
        "Hdr": {
          "Name": "example.test.",
          "Rrtype": 15,
          "Class": 1,
          "Ttl": 60,
          "Rdlength": 9
        },
        "Preference": 10,
        "Mx": "mail.example.test."
      }
    ],
    "ID": 34374,
    "Truncated": false
  },
  {
    "Server": "127.0.0.11:53",
    "QueryTime": 2000000,
    "Answers": [
      {
        "Hdr": {
          "Name": "example.test.",
          "Rrtype": 16,
          "Class": 1,
          "Ttl": 60,
          "Rdlength": 15
        },
        "Txt": [
          "v=spf1 mx -all"
        ]
      }
    ],
    "ID": 3010,
    "Truncated": false
  },
  {
    "Server": "127.0.0.11:53",
    "QueryTime": 2000000,
    "Answers": [
      {
        "Hdr": {
          "Name": "example.test.",
          "Rrtype": 1,
          "Class": 1,
          "Ttl": 60,
          "Rdlength": 4
        },
        "A": "172.16.42.11"
      }
    ],
    "ID": 57132,
    "Truncated": false
  },
  {
    "Server": "127.0.0.11:53",
    "QueryTime": 2000000,
    "Answers": [
      {
        "Hdr": {
          "Name": "example.test.",
          "Rrtype": 2,
          "Class": 1,
          "Ttl": 60,
          "Rdlength": 6
        },
        "Ns": "ns1.example.test."
      }
    ],
    "ID": 22482,
    "Truncated": false
  }
]

doggo output is a bit nicer with all lowercase field names and more predictable layout (answers.address provides the record value, and answers.type the record type, I don't need to have a mapping of Answers.Hdr.Rrtype value to Answers[$RECORD_TYPE]), so it's a bit easier to process:

Doggo JSON output

[
    {
        "answers": [
            {
                "name": "example.test.",
                "type": "NS",
                "class": "IN",
                "ttl": "60s",
                "address": "ns1.example.test.",
                "status": "",
                "rtt": "1ms",
                "nameserver": "127.0.0.11:53"
            }
        ],
        "authorities": null,
        "questions": [
            {
                "name": "example.test.",
                "type": "NS",
                "class": "IN"
            }
        ]
    },
    {
        "answers": [
            {
                "name": "example.test.",
                "type": "MX",
                "class": "IN",
                "ttl": "60s",
                "address": "10 mail.example.test.",
                "status": "",
                "rtt": "0ms",
                "nameserver": "127.0.0.11:53"
            }
        ],
        "authorities": null,
        "questions": [
            {
                "name": "example.test.",
                "type": "MX",
                "class": "IN"
            }
        ]
    },
    {
        "answers": [
            {
                "name": "example.test.",
                "type": "TXT",
                "class": "IN",
                "ttl": "60s",
                "address": "\"v=spf1 mx -all\"",
                "status": "",
                "rtt": "0ms",
                "nameserver": "127.0.0.11:53"
            }
        ],
        "authorities": null,
        "questions": [
            {
                "name": "example.test.",
                "type": "TXT",
                "class": "IN"
            }
        ]
    },
    {
        "answers": [
            {
                "name": "example.test.",
                "type": "A",
                "class": "IN",
                "ttl": "60s",
                "address": "172.16.42.11",
                "status": "",
                "rtt": "0ms",
                "nameserver": "127.0.0.11:53"
            }
        ],
        "authorities": null,
        "questions": [
            {
                "name": "example.test.",
                "type": "A",
                "class": "IN"
            }
        ]
    }
]

That's not an issue specific to JSON format though, YAML is affected by that too and I guess changing that would be a breaking change for anyone relying on the current output 😅

q does at least have some improvement with how MX record splits out the preference field. For doggo splitting a string by the space char as a delimiter isn't difficult if separating the preference was required.

JSON vs YAML format inconsistency with field names

JSONL aside, the JSON output is using PascalCase convention for fields, while YAML output is using flatcase.

Additionally, the YAML output has each result split by a blank line, which isn't valid YAML?

• --- can be used to specify multiple YAML docs.
• An array of results by default would be better?

[polarathene]

jq / yq manipulation (eg: Aligned Columns output)

With doggo output, you can easily produce the aligned columns of your choice too (picks two fields of interest and outputs as TSV document):

# The sub operator is only to remove escaped double quote wrapping of TXT values
# Can omit, but output between tools for TSV is then inconsistent for TXT values

# jq
doggo example.test NS MX TXT A --json | jq -r '["TYPE", "VALUE"], (.[] | [.answers[0].type, .answers[0].address])
  | .[1] |= sub("\""; ""; "g")
  | @tsv'

# yq
doggo example.test NS MX TXT A --json \
  | yq -o=tsv '[["TYPE", "VALUE"]] + [.[]
    | [.answers[0].type, .answers[0].address]
    | .[1] |= sub("\"", "")
    ]'

TYPE    VALUE
NS      ns1.example.test.
MX      10 mail.example.test.
TXT     v=spf1 mx -all
A       172.16.42.11

I'm not sure how you'd approach that with q, but assume it's a bit more tricky? 🤷‍♂️

---

EDIT: I figured it out for q, but as you can see below it's much more work 😞

Steps

Process used for YAML format (JSONL is same but skips step 1):

1. Take the YAML output, convert blank lines to YAML mult-doc separator ---, then remove the last line (avoid empty doc) (head -n -1 before sed is not needed, step 6 filters out the empty array).
2. Use reduce to merge multi-doc into array of results (which would be nicer to have to begin with?).
3. Filter the content to answers field and exclude the hdr section.
4. Rewrite MX items to concatenate the mx + preference fields with space delimiter into the mx field and remove the preference field.
5. Rewrite TXT items array elements into string.
6. Transform document structure from - key: value into - [key, value] as input required for outputting TSV.
7. Next transform won't work as we're still operating on a stream of separate documents. We stop here to output the YAML as a single doc input.
8. The 2nd yq command now ingests that single YAML doc and prepends the TSV headers, then outputs as TSV 🎉

YAML

q example.test NS MX TXT A --format yaml \
  | sed 's/^$/---/' \
  | yq '. as $item ireduce ({}; [$item])
    | .[].answers[] | [with_entries(select(.key != "hdr"))]
    | with(select(.[] | has("mx")); .[] |= (.mx = ([.preference, .mx] | join(" "))) |= del .preference)
    | with(select(.[] | has("txt")); .[].txt |= join(""))
    | (.[] | to_entries | with(.[]; . |= [.key | upcase, .value]))' \
  | yq -o=tsv '[["TYPE", "VALUE"]] + .'

JSONL (--format json)

Here's the equivalent for the JSONL output from q. yq treats it as a multi-doc so no sed needed. Other than that it's just adjusted to the PascalCase convention:

q example.test NS MX TXT A --format json \
  | yq -p=json '. as $item ireduce ({}; [$item])
    | .[].Answers[] | [with_entries(select(.key != "Hdr"))]
    | with(select(.[] | has("Mx")); .[] |= (.Mx = ([.Preference, .Mx] | join(" "))) |= del .Preference)
    | with(select(.[] | has("Txt")); .[].Txt |= join(""))
    | (.[] | to_entries | with(.[]; . |= [.key | upcase, .value]))' \
  | yq -o=tsv '[["TYPE", "VALUE"]] + .'

[natesales]

Thanks for the fabulously detailed feedback! v0.19.0 brings some changes:

• All "inverted" flags (with the exception of tls-insecure-skip-verify) have been reversed to become default true booleans. They can be toggled with the dig syntax of +noflag or --flag=false. I understand this is a departure from convention for certain CLI programs, but I think this strikes a balance of familiarity with DNS utilities and a more standardized unix flag format.

• Pretty TTLs are now truncated when the minute or second component(s) are zero.

• The structured output (JSON and YAML) format is now a list of output.Entrys with all lowercase field names. This is consistent between JSON and YAML.

• There is a new column output format:

$ q --format=column example.com
   A 1h29m33s 93.184.216.34
AAAA 52m42s   2606:2800:220:1:248:1893:25c8:1946
  MX 9h25m2s  0 .
  NS 3h21m31s a.iana-servers.net.
  NS 3h21m31s b.iana-servers.net.
 TXT 9h25m2s  "v=spf1 -all"
 TXT 9h25m2s  "wgyf8z8cgvm2qmxpnbnldrcltvk4xqfn"

[polarathene]

Thanks for the fabulously detailed feedback!

Brilliant! Thanks for tackling the concerns raised ❤️

I'm not able to give it a spin right now but I look forward to it! 😁

[natesales]

Closing for lack of activity, feel free to reopen if needed.