Command line tutorial - splitting files into smaller chunks

  • June 3, 2007
  • Avatar for peter
    Peter
    Upfold

I'm sure we have all come across the situation - you have a really big file you want to transfer and the only way you can transfer the file with the setup you have doesn't have enough space. Or, you might want to archive some backups on CD or DVD, for example, but the files are too big.

Built into your Linux distribution (or other Unix OS) among the arsenal of command line tools is a command called split. As the name suggests, it's a simple tool that creates many files from one file and distributes the data in them.

To run split, you need to tell it the size of each chunk you want and the source file (and there are several other options you can pass to it as well):

$ split -b 1024 filetosplit

The number after the -b switch is the number of bytes for each chunk. You can set the number to be interpreted as a number of kilobytes or megabytes by adding a lowercase k or m, respectively, after the number (no space!).

By default, split will spit out files named like xaa, xab and so on into the current directory. The x is a prefix and then the two letters can be used to identify the correct sequence to stick the files back together later.

If you want to use a custom prefix instead of x, simply tack that onto the end of the command:

$ split -b 500m bigfiletosplit prefix

You should now have your split files. The important part is that you need to be able to put them back together!

Thankfully, you're covered there as well, with cat. Simply run the following command (replace x with your custom prefix if you've set one).

$ cat x?? > recreatedfile

Once it's finished putting all the bits together (which could take sometime depending on your file size), you'll have a recreated file again!

Avatar for peter Peter Upfold

Home » Articles »