Dealing with charset=utf-8 in application/json responses

[bernhardreiter]

Split out from #588:

How should we deal with a charset=utf-8 parameter to the content_type: application/json header in responses? This parameter is not allowed by the current JSON standard (rfc8259).

While rfc9431 recommends to be strict when receiving a non-standard thing for mid term benefits, we also should consider the ease of deployment.

After talks to @s-l-teichmann and @tschmidtb51, the plan is to

• check out how popular webservers like freenginx, Apache and IIS would serve a .json file from disk by default.
• find instructions to easily change this default to something else.

Depending on this we may consider to accept those responses for the downloader and aggregator and be strict on the checker, or be strict with both.

[adulau]

I think we have to be realistic: there are many poorly implemented systems out there serving JSON with a charset defined in the Content-Type header. Silently dropping it is a more sensible approach than being overly strict about this.

[tschmidtb51]

@adulau I'm currently in favor of reporting it in the csaf_checker as error but warn in csaf_downloader (and accept the file). With this approach, we add point out that something is not right and still make it usable for the consumer that usually has no power of the provider's webserver settings...

[bernhardreiter]

: there are many poorly implemented systems out there serving JSON with a charset defined in the Content-Type header.

Do you know which systems do so by default?