Should JSONResponse add "charset=utf-8" to "Content-Type" header?

[sashkent3]

For example, Dart clients as described here will by default use latin1 charset to comply with RFC 2616. Considering that JSONResponse explicitly encodes it's body using utf8 it might be a good idea to let the client know it.

[jhominal]

The encoding content type parameter is defined only for text MIME subtypes, but the standard JSON media type is an application subtype, application/json:

• RFC 2046 defines charset as part of the specification for the text type;
• RFC 2616 sets the default of iso-8859-1 only for text subtypes (emphasis mine):

  When no explicit charset parameter is provided by the sender, media subtypes of the "text" type are defined to have a default charset value of "ISO-8859-1" when received via HTTP.

• RFC 9110, that supersedes RFC 2616, says (emphasis mine):

  HTTP uses "charset" names to indicate or negotiate the character encoding scheme ([RFC6365], Section 2) of a textual representation.

• RFC 4627, that defines the application/json media type, requires that JSON be encoded in unicode, and does not make any provision for an encoding parameter in the media type;

  JSON text SHALL be encoded in Unicode.

In other words, I believe that means that no charset parameter should be set by JSONResponse.

As for the behavior of Dart's http/Response.body property, that behavior seems to be designed to handle text subtypes - it would be inappropriate to use it to process e.g. image, binary, XML (which will generally put charset information in the XML declaration), or JSON. In these cases, you should use http/Response.bodyBytes and pass that result to the appropriate parser.

[sashkent3]

Got it, thanks for the clarification. Is it safe to assume utf-8 charset for JSONs then?

[jhominal]

According to the spec one has to check which Unicode encoding it is - it can be utf16 or utf32 (though I have never seen that)

But why do you ask that question?

[sashkent3]

I'm designing a system currently, both frontend and backend. And although I have total control over behavior of both ends I would like to design it in a universal (as in pluggable) way. It's nice to choose a widely accepted standard so I no longer should spend time thinking about it in that particular project and ever again.

[jhominal]

All right, then I will say:

• theoretically you should not assume that JSON will be utf8, it can be another form of unicode encoding (utf16, utf32) - however, that management should be handled by JSON parsers (similarly to how XML parsers have to use the XML declaration to select the encoding);
• in practice, I suspect that nearly all JSON anywhere is emitted as utf8;
• and if you emit JSON (whether as client or server), definitely emit it as utf8;