I know it’s really hard to believe, but not so very long ago, websites and apps like Spotify did not exist. Music collections were purchased CD by CD, cassette by cassette, and album by album. Before the iPod but after the cassette mixtape, car music systems could handle custom-built CDs, but they were quirky. Some, like the one I had, could only read from the root directory but could handle thousands of songs in that directory.

Like most people who were ripping their music to MP3 in that era, I meticulously kept my music in deep directory structures, separated by artist and album. Copying each song, or even a directory of songs, to a flat directory where I could then burn them to a CD would have been a long process. Also, many artists have songs by the same name as others, so duplicates were also an issue. Thus, copyFlat was born.

You might be wondering why I would share a utility I wrote from a bygone era (2007). There are several reasons I think this code is still useful today. First, perhaps there are other situations where one might want to take a deep directory structure and flatten it. Second, and really most of all, there are computer science techniques in this small set of code that may be of interest to Java programmers learning how to code, such as recursion, threading, and file manipulation.

Getting Set Up

This code compiles and runs on any version of Java (including Java 8 and beyond). It requires no extra libraries or classpaths, making it a nice example of basic Java programming. Since I have an Apple M1 chip, I tested it on the following Java versions:

• OpenJDK Zulu 8, OpenJDK Runtime Environment (Zulu 8.76.0.17-CA-macos-aarch64) (build 1.8.0_402-b06)
• OpenJDK Temurin 11, OpenJDK Runtime Environment Temurin-11.0.19+7 (build 11.0.19+7)

I have updated the code since I originally wrote it in 2007 to include using try-with-resources, so all open files, streams, and channels will automatically close.

Requirements

The requirements I set out to meet were fairly mundane. I came up with three in total.

1. I wanted to copy files from a deeply nested directory structure to a newly created flat directory, renaming files that would be duplicates in the process.
2. I also wanted a song list generated in a text file, skipping any artwork or other files copied along the way.
3. Finally, I wanted the application to update progress in the terminal as it was working.

I’m going to list the file in its entirety here, and then we’ll discuss what is going on in the sections that follow.

/*
 * copyFlat.java
 *
 * Created on May 11, 2007, 8:29 PM
 * Updated on Mar 10, 2024
 */

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.PrintWriter;
import java.nio.channels.FileChannel;
import java.util.ArrayList;
import java.util.List;

/**
 *
 * @author John Boardman
 */
public class copyFlat implements Runnable {
    private static int filesGenerated = 0;
    private static String fromDir = "", toDir = "", fileSuffix = ".*.*";
    
    public static void main(String[] args) {
        if (args.length < 2) {
            System.out.println("Usage: copyFlat fromDir toDir [filesuffix]\nExample: copyFlat c:/srcdir c:/destdir mp3");
            System.exit(-1);
        }

        fromDir = args[0];
        toDir = args[1];
        if (args.length > 2) {
            fileSuffix = args[2];
            if (fileSuffix != ".*.*") {
                fileSuffix = "(.*)" + fileSuffix;
            }
        }

        ThreadGroup tg = new ThreadGroup("gen");
        Thread t = new Thread(tg, new copyFlat());
        t.start();

        while (tg.activeCount() > 0) {
            try {
                Thread.sleep(1000);
            } catch (InterruptedException ex) {
                ex.printStackTrace();
            }
        }
    }

    public void run() {
        try {
            File delLogFile = new File("songlist.txt");
            delLogFile.delete();
        } catch( Exception e) {}

        try (PrintWriter logFile = new PrintWriter("songlist.txt")) {
            List<File> files = new ArrayList<File>(2048);
            findFiles(fromDir, null, files);
            filesGenerated = 0;

            if (files.size() > 0) {
                createAndValidateDir(toDir);
            }

            while (filesGenerated < files.size()) {
                File f = (File)files.get(filesGenerated);
                if (f.getAbsolutePath().endsWith(".mp3")) {
                    String source = f.getAbsolutePath();
                    source = source.replace('\\', '/');
                    String dest = "";
                    String[] dirs = source.split("/");
                    if (dirs != null && dirs.length > 2) {
                        dest = dest.concat(dirs[dirs.length - 3])
                                   .concat("-")
                                   .concat(dirs[dirs.length - 2])
                                   .concat("-")
                                   .concat(dirs[dirs.length - 1]);
                    }

                    copyFile(source, toDir + "/" + dest);
                    logFile.println(dest);
                } else {
                    String source = f.getAbsolutePath();
                    source = source.replace('\\', '/');
                    String dest = toDir.concat("/").concat(f.getName());
                    dest = dest.replace('\\', '/');
                    copyFile(source, dest);
                }

                ++filesGenerated;
                System.out.println(new StringBuffer()
                  .append("copied ")
                  .append(filesGenerated)
                  .append("/")
                  .append(files.size())
                  .append(" files").toString());
            }
        } catch (Exception ex) {
            StringBuffer sb = new StringBuffer(1024);
            sb.append(ex.getMessage()).append("\n");
            StackTraceElement[] stackTrace = ex.getStackTrace();
            for (int st = 0; st < stackTrace.length; st++) {
                sb.append(stackTrace[st].toString()).append("\n");
            }

            sb.append("\naborted due to error!");
            System.out.println(sb.toString());
        }
    }

    private void copyFile(String source, String dest) throws IOException {
        try (FileInputStream srcInputStream = new FileInputStream(source);
             FileChannel srcChannel = srcInputStream.getChannel();
             FileOutputStream dstOutputStream = new FileOutputStream(dest);
             FileChannel dstChannel = dstOutputStream.getChannel()) {

            // Copy file contents from source to destination
            dstChannel.transferFrom(srcChannel, 0, srcChannel.size());
        }
    }

    /**
     * Find all files we want to copy
     */
    protected static void findFiles(String startDir, String currentDir, List<File> fileList) throws Exception {
        // start case
        if (currentDir == null) {
            currentDir = startDir;
        // end case
        } else if (currentDir.equals(startDir)) {
            return;
        }
        
        File dir = new File(currentDir);
        if (dir.exists()) { 
            File[] files = dir.listFiles();
            if (files != null && files.length > 0) {
                for (int i = 0; i < files.length; i++) {
                    File f = files[i];
                    if (f.isFile()) {
                        if (f.getName() != null) {
                            if (f.getName().matches(fileSuffix)) {
                                fileList.add(f);
                            } else {
                                System.out.println("Did not copy file ".concat(f.getName()));
                            }
                        }
                    } else if (f.isDirectory()) {
                        findFiles(startDir, f.getAbsolutePath(), fileList);
                    }
                }

                files = null;
            }
        }
    }

    /**
     * Replace all occurrences of strRemove with strAdd in strSource.
     *
     * @param strSource - the source string
     * @param strRemove - the substring to replace
     * @param strAdd - the replacement substring
     * @return a new string
     */
    public static final String replace(String strSource, String strRemove, String strAdd) {
       int iRemoveLen = strRemove.length();
       int iAddLen = strAdd.length();
       int iRemovePos = -1;

       while (-1 != (iRemovePos = strSource.indexOf(strRemove, iRemovePos))) {
          strSource = strSource.substring(0, iRemovePos) + strAdd +
                      strSource.substring(iRemovePos + iRemoveLen);

          iRemovePos += iAddLen;
       }

       return strSource;
    }

    /**
     * @param path path to create if it does not exist
     * @return true if directory had to be created, false if it was already there
     * @throws IOException 
     */
    protected boolean createAndValidateDir(String path) throws IOException {
        File dir = new File(path);
        if (dir.getParentFile() != null && !dir.getParentFile().exists()) {
            if (!dir.getParentFile().mkdirs()) {
                throw new IOException("Could not create destination directory! ".concat(dir.getParent()));
            }
            
            return true;
        }
        
        return false;
    }
}

So that’s the whole file, which is also the entire application. Now, let’s go method by method through the file.

main, like in all Java classes that are executable, is the entry point into our application. This is a very simple main that checks arguments, creates a thread, starts it, and waits for it to finish. While this application could have been written without using threads, I like to keep long running processes out of the main thread.

The only thing really to note here is the fileSuffix parameter, which is optional. The matches() method on String takes a bit of a weird regular expression. ".*.*" means “match anything”. If you want to match suffixes, you have to use syntax like "(.*)" + fileSuffix, which means “match anything ending with the fileSuffix”.

This is the main method for the thread. The first thing it does is remove any existing songlist.txt file since we don’t want to append our new songlist to any existing one. We don’t care if one did not exist before, so we just eat any exception.

After that, we make sure the directory we are copying to exists, and if not, we create it. Next, we use the try-with-resources syntax to open a new songlist.txt file.

After finding the list of files we want to copy and storing them in an array, we loop through all of those files and copy them to the new directory. To avoid duplicate song names, if we encounter an mp3 file, we use the outer directory names (artist/album) to construct a new name separated by hyphens.

Finally, we report the current status to the user.

copyFile

This short method is really the meat of the application. Again using the try-with-resources syntax, we open our input and output streams and channels. Then, we call transferFrom(), which is a synchronous function, to copy the file all at once.

The documentation for this method indicates that it doesn’t have to copy everything (or anything), but I’ve never had it do anything except copy the entire file correctly, whether it is binary or text.

findFiles

This is the recursive method (Java recursion in action) that winds its way through the filesystem structure, building a list of files to copy as it goes. It makes heavy use of the Java File capabilities to figure out what are directories vs. files.

If a suffix parameter was provided, it filters only those files that match the suffix. When a directory is reached, it recurses into it. When all files and directories have been examined, the fileList parameter will contain the list of files we are interested in copying.

replace

This method is unlike String.replaceAll because it does not use regular expressions to find the characters to replace. Otherwise, it’s just a straightforward “find and replace in a string” utility method.

createAndValidateDir

Given a path, this method ensures that the path either exists or can be successfully created. Otherwise, the application ends because it cannot copy any files to an invalid destination.

Conclusion

By using Java’s built-in file classes, along with recursion, it turns out to be pretty easy to implement the requirements that I came up! It’s fairly simple to copy a set of files nested inside a directory structure into a new flat directory.

Remember, while you probably aren’t burning your own CDs or cassettes nowadays, there are reasons why the concepts demonstrated here are still relevant. The computer science techniques in this small set of code (such as recursion, threading, and file manipulation) are basic skills all Java programmers should know.

I hope something along the way has been informative, new, and/or interesting. Feel free to take this code and use it however you wish to create your own utilities, and subscribe to the Keyhole Dev Blog for more.