Approach: implement a deterministic state-machine parser that reads characters once, supports quoted fields ("" -> "), embedded commas/newlines, empty input, irregular row lengths, and arbitrarily long fields (no fixed buffer). Return list of rows (each row is list of strings). For streaming, expose a generator that yields rows as they complete.python
def parse_csv(s):
"""
Parse CSV string into list of rows (each row is a list of fields).
Handles quoted fields with commas/newlines, escaped quotes (""), empty input,
irregular columns, and arbitrarily long fields.
"""
if s is None:
return []
rows = []
row = []
field = []
i = 0
n = len(s)
IN_FIELD, IN_QUOTED, AFTER_QUOTE = 0, 1, 2
state = IN_FIELD
while i < n:
ch = s[i]
if state == IN_FIELD:
if ch == '"':
state = IN_QUOTED
elif ch == ',':
row.append(''.join(field)); field = []
elif ch == '\n':
row.append(''.join(field)); rows.append(row); row = []; field = []
elif ch == '\r':
# handle CRLF or lone CR
if i+1 < n and s[i+1] == '\n':
i += 1
row.append(''.join(field)); rows.append(row); row = []; field = []
else:
field.append(ch)
elif state == IN_QUOTED:
if ch == '"':
state = AFTER_QUOTE
else:
field.append(ch)
else: # AFTER_QUOTE
if ch == '"': # escaped quote inside quoted field
field.append('"'); state = IN_QUOTED
elif ch == ',':
row.append(''.join(field)); field = []; state = IN_FIELD
elif ch == '\n':
row.append(''.join(field)); rows.append(row); row = []; field = []; state = IN_FIELD
elif ch == '\r':
if i+1 < n and s[i+1] == '\n':
i += 1
row.append(''.join(field)); rows.append(row); row = []; field = []; state = IN_FIELD
elif ch.isspace():
# ignore spaces between closing quote and delimiter/newline (tolerant)
# remain in AFTER_QUOTE
pass
else:
# malformed: treat as part of field (lenient) or raise error.
# We'll append remaining char and switch to IN_FIELD for leniency.
field.append(ch); state = IN_FIELD
i += 1
# end of input: flush
if state == IN_QUOTED:
# Unterminated quoted field: leniently accept field as-is
row.append(''.join(field))
else:
row.append(''.join(field))
# if last row is just an empty row at EOF due to trailing newline, keep it
if row != []:
rows.append(row)
return rows
# Example usage:
# parse_csv('a,"b,with,commas","c\nwith\nnewlines","escaped ""quote"""')
Key concepts:- Single-pass state machine: IN_FIELD, IN_QUOTED, AFTER_QUOTE.- Handles CRLF and lone CR, tolerates spaces after closing quote.- Uses list buffers to support very long fields (no size limits in Python aside from memory).Time complexity: O(n) where n = length of input (single pass).Space complexity: O(n) for result and field buffers.Edge cases to consider:- Empty string -> []- Single empty field: "" -> [['']]- Trailing comma -> row with empty last field- Irregular column counts across rows -> allowed; caller can validate schema- Unterminated quoted field -> currently lenient; could raise error depending on strictnessTests for malformed inputs:- Unterminated quotes: 'a,"b,c' — expect error or lenient parse- Escaped quotes malformed: '"a""b' etc.- Characters after closing quote before comma: '"a"x,b' -> lenient vs strict- Mixed CR/LF patterns: CR alone, LF alone, CRLFStreaming scenarios:- Provide generator variant that accepts an iterator of chunks (e.g., file.read(CHUNK)) and preserves parser state between chunks, yielding rows when complete. Important tests: field split across chunks, quote boundary at chunk edge, extremely long field spanning many chunks. Alternative: use Python's csv module for production-grade needs.