This content originally appeared on DEV Community and was authored by Mateen Kiani
Introduction
Storing data is one of the most common tasks you’ll tackle in Python. You might generate a list of user names, numerical results, or configuration values that you later need to reuse or share. One often overlooked aspect is choosing the right file format and method to save that list in a way that stays maintainable and easy to read.
But how do you decide whether to write a list as plain text, CSV, or JSON—and what’s the best practice for large datasets?
In this article, we’ll walk through three popular approaches: saving as plain text, exporting to CSV, and serializing to JSON. You’ll see code examples, tips, and practical guidance on picking the right option for your project.
Write as Plain Text
Writing a Python list directly to a text file is straightforward. You get a human-readable file that you can open in any editor. Here’s a simple example:
my_list = ["apple", "banana", "cherry"]
with open("fruits.txt", "w", encoding="utf-8") as f:
for item in my_list:
f.write(item + "\n")
Each element ends up on its own line. Pros of this method:
- Easy to read in any text editor
- No extra dependencies
- Good for small or simple lists
However, plain text has limitations:
- No structure for nested lists or dictionaries
- Hard to parse back into Python data types without custom code
- Lines may need trimming of newline characters
Tip: If you also work in Node.js, check out Writing to files in Node.js to see a similar approach in JavaScript.
Save as CSV
Comma-separated values (CSV) is ideal when your list contains simple records of the same length—for example, rows of numeric data or fixed fields.
import csv
rows = [["Name", "Age", "City"],
["Alice", 30, "New York"],
["Bob", 25, "Paris"]]
with open("people.csv", "w", newline='', encoding="utf-8") as csvfile:
writer = csv.writer(csvfile)
writer.writerows(rows)
CSV benefits:
- Widely supported by spreadsheets and databases
- Easy import/export for tabular data
- Handles quoting and special characters automatically
Drawbacks:
- Not ideal for nested or complex data structures
- Requires consistent row structure
Tip: Use
newline=''
in Python 3 to avoid adding extra blank lines on Windows.
Use JSON Format
JSON shines when your list may include nested lists or dictionaries. It preserves data types and structure, making it easy to load back into Python.
import json
my_data = [
{"id": 1, "name": "Alice"},
{"id": 2, "name": "Bob"}
]
with open("data.json", "w", encoding="utf-8") as f:
json.dump(my_data, f, indent=4)
Advantages:
- Maintains nested structures
- Built-in support for lists, dicts, strings, numbers, booleans
- Readable with
indent
Considerations:
- File size can be larger than CSV or text
- Requires
json
module (built-in)
Tip: Use
json.loads()
andjson.dumps()
for working with strings instead of files.
Handle Large Lists
When your list has millions of items, consider streaming or chunked writes to avoid high memory usage.
def chunked_writer(filename, iterable, chunk_size=10000):
with open(filename, 'w', encoding='utf-8') as f:
for i in range(0, len(iterable), chunk_size):
chunk = iterable[i:i + chunk_size]
f.write("\n".join(str(x) for x in chunk) + "\n")
big_list = range(1_000_000)
chunked_writer("big_data.txt", big_list)
Key points:
- Process data in slices to limit peak memory
- Choose a chunk size that balances I/O overhead and memory overhead
- Monitor disk performance when writing large files
Best Practices and Tips
- Always use a
with
block for file operations to ensure proper closure. - Specify
encoding='utf-8'
to avoid character encoding issues. - Clean and validate your list items before writing to prevent injection or formatting errors.
- Use file extensions (
.txt
,.csv
,.json
) that match the format for clarity.
Best Practice: Close files automatically and handle exceptions. You can wrap writes in a
try-except
block or use context managers to keep code clean.
Conclusion
Choosing the right way to save a Python list to a file depends on your data’s structure and how you plan to retrieve it. Plain text is quick and human-readable, CSV works great for tabular data, and JSON lets you preserve nesting and data types. For very large lists, chunk your writes to keep memory use in check.
With these methods and tips, you’re ready to pick the best file format for your next project and avoid surprises down the road. Happy coding!
This content originally appeared on DEV Community and was authored by Mateen Kiani