# suno-data-processing
Data processing for suno


## Run in oracle cloud
```
python install_python_deps.py
python build_cpp_parser.py
# this input and output dir are default, you can ust run ./parser --in-memory
./parser --in-memory --input /tmp/test_data.jsonl --output-dir /mnt/localdisk/parser-output  
```


## Run in your local mac machine
```
python install_python_deps.py 
python build_cpp_parser.py
# this is slower because local don't have that much memory
./parser --input /tmp/test_data.jsonl --output-dir ./
```



## Python notebook usage
```
!pip install orjson tqdm matplotlib pyarrow polars
import pyarrow as pa
import polars as pl
import pyarrow.ipc as ipc
with pa.memory_map("/mnt/localdisk/parser-output/parsed_results_2025-03-14_12-46.arrow", 'r') as source:
    reader = ipc.open_file(source)
    table = reader.read_all()
df = pl.from_arrow(table)
df.head()
```