split

The split command divides a CSV file into multiple smaller files based on various criteria.

Syntax

qsv split [<options>] [<input_file>]

The split command splits the input file (or stdin if no file is specified) into multiple smaller files based on the specified options.

-n <num_lines>: Split the file into smaller files with a maximum of <num_lines> lines each.
--size <size>: Split the file into smaller files with a maximum size of <size> bytes each.
--kb-size <size>: Split the file into smaller files with a maximum size of <size> kilobytes each.
-s <num_lines>: Split the file into smaller files with a maximum of <num_lines> lines each, padding the file names with zeros.
--filename <pattern>: Use <pattern> as the file name pattern for the output files.
--pad <num_digits>: Pad the file names with <num_digits> zeros.
<outdir>: The directory where the output files will be written.

Split a file into smaller files with 1000 lines each:

qsv split -n 1000 input.csv

Split a file into smaller files with a maximum size of 100 bytes each, using a custom file name pattern:

qsv split outdir --size 100 --filename chunk_{}.csv input.csv

Split a file into smaller files with a maximum of 100 lines each, using padding and writing to a subdirectory:

qsv split outdir/subdir -s 100 --filename chunk_{}.csv --pad 5 input.csv

Split a file into smaller files with a maximum of 100 lines each, writing to the current directory:

qsv split . -s 100 input.csv

Split a file into smaller files with a maximum size of 1000 kilobytes each:

qsv split outdir --kb-size 1000 input.csv

Splitting large files for easier processing or to meet file size limits for uploading to a database
Organizing data into a hierarchical directory structure
Creating smaller files for more efficient data processing or analysis