JSON Response Filtering Fails with Non-ASCII Characters

WantChane ·a year ago

ReleasedBug

from flask import Flask, jsonify
app = Flask(__name__)

DATA = {"message": {"en-us": "Hello World!", "zh-cn": "你好，世界！"}}

@app.route('/hello_ascii')
def hello_ascii():
    app.json.ensure_ascii = True
    return jsonify(DATA)

@app.route('/hello_no_ascii')
def hello_no_ascii():
    app.json.ensure_ascii = False
    return jsonify(DATA)

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)

JSONPath filter: $.message.zh-cn

hello_ascii

before:

{ "message": { "en-us": "Hello World!", "zh-cn": "你好，世界！" } }

after:

[ "你好，世界！" ]

hello_no_ascii

before:

{ "message": { "en-us": "Hello World!", "zh-cn": "你好，世界！" } }

after:

[ "菴�螂ｽ�御ｸ也阜��" ]

Windows2025.5.5

Comments (7)

Gregory Schier

·a year ago

Oh, that’s not good. Thanks reporting this!

Hao Xiang·10 months ago

Reproduced.

It is caused by the charset used when filtering json.

The first step of filter is loading the content from reponse body with a specificed decoder.

When json.ensure_ascii = True, the reponse body are escaped with \uXXXX sequences. So the decoder works. But app.json.ensure_ascii = False fails it.

The root reason is that the loading decoder does’t support other charset just SHIFT_JIS.

from flask import Flask, make_response

app = Flask(__name__)

DATA = '''{"message": {"en-us": "Hello World!", "zh-cn": "你好，世界！"}}'''


def make_response_with_charset(data, charset):
    response = make_response()
    response.data = data.encode(charset)
    response.mimetype = 'application/json'
    response.mimetype_params.update({'charset': charset})
    return response

@app.route('/hello_utf8')
def hello_ascii():
    return make_response_with_charset(DATA, 'utf-8')

@app.route('/hello_gb2312')
def hello_no_ascii():
    return make_response_with_charset(DATA, 'gb2312')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)

try fix by

https://github.com/mountain-loop/yaak/pull/251