I am trying to use ast.literal_eval() to safely convert strings into Python objects like lists or dictionaries. However, I keep encountering the error ValueError: malformed node or string. I thought this function was a safer alternative to eval(), but it seems to fail on strings that I expect to be valid. What are the common causes for this—such as passing a non-string object, including mathematical operations like 1+2, or using JSON-style values like true, false, or null? Should I be using json.loads() instead, or is there a way to sanitize my strings so that ast.literal_eval() can process them correctly?
3 answers
The "malformed node or string" error occurs because ast.literal_eval() is strictly designed to evaluate Python literals only (strings, numbers, tuples, lists, dicts, sets, booleans, and None). If your string contains anything else—like a variable name, a function call (datetime.now()), or a math operation (4 + 5)—it will fail. It is not a general-purpose expression evaluator like eval(). To fix this, ensure your input string strictly follows Python syntax. If you are dealing with data from a web API, you are likely actually looking at JSON, which uses true (lowercase) and null instead of Python's True and None. In that case, switch to json.loads(your_string) for a much smoother experience.
[Image showing the difference between a Python Literal (valid) and a Python Expression (invalid for ast.literal_eval)]
I noticed that I get this error even when I pass a string that looks like a valid dictionary. It turns out I was accidentally passing the dictionary object itself instead of its string representation! If you run ast.literal_eval({"a": 1}), it crashes because it expects a str. Always ensure you are wrapping your data in quotes or using str(my_data) if you're testing it.
If you absolutely need to evaluate math (like "1+2") safely without using the dangerous eval(), you can't use ast.literal_eval(). You'll need a specialized library like simpleeval or a custom AST walker.
This is a great distinction, Linda. While ast.literal_eval is excellent for safety, its strictness is its main limitation. For anything dynamic, like a calculator app or complex configuration parsing, we often suggest using the numexpr library or building a custom NodeTransformer to only allow specific arithmetic nodes.
Steven, you're spot on. Another sneaky cause is having null in your string. I often fix this by doing ast.literal_eval(my_str.replace("null", "None")), though as Patricia mentioned, if you're doing that much replacement, json.loads() is usually the "cleaner" Software Development choice.