Remove the entire content of a folder in storage

[halindraprakoso]

Hello,

is it possible to delete the folder or entire content of a folder using storage.remove()?

[GaryAustin1]

No. You can empty a bucket with a call or use the UI.
Otherwise you have to build up a list of file paths from the "folder" and remove them in a loop or as an array. I'm not sure how well that would work for large amount of files in a single call with an array, or looping in a database function. You probably need to run such code in a cloud function or on your server if you have one.
The new studio open source likely has examples of deleting all files in a folder as the user interface does that for you.

[halindraprakoso]

yeah, I thought it would work with wildcard like this:

const { error: removeError } = await supabase.storage
			.from(bucket)
			.remove([`${user_id}/${folderName}/*`]);

I can do this instead:

const { data:list, error } = await supabase.storage.from(bucket).list(`${user_id}/${folderName}`);
const filesToRemove = list.map((x) => `${user_id}/${folderName}/${x.name}`);

const { data, error } = await supabase.storage.from(bucket).remove(filesToRemove);

[GaryAustin1]

I have never seen the * method work. Be careful if you have a lot of files. Besides server load, if .remove passes data in the url versus body like post, you will blow up url length quickly. I don't know how Supabase storage API implements DELETE as body is optional. Normally all resources are in the url for DELETE.

[halindraprakoso]

fortunately, I won't be dealing with a large number of files for this case but that's an interesting point to remember!

[logemann]

The UI can also delete complete folders... so do they also iterate through the files and then delete the folder itself?

[GaryAustin1]

Yes, last time I looked at the source code (6 months ago) that is what the UI does. The folder is not a real thing in s3 storage. In the UI it is just a placeholder object with a special name and used only to help display things.

[logemann]

yeah you are right. Just tested it and still need to iterate files. And of course, a folder is just a simple object in S3 and therefore same mechanics apply to supabase.

[bart]

Problem with the solution from above are folders inside folders. Supabase doesn't provide a way for recursive listing as far as I know so we have to implement it manually, right?

[GaryAustin1]

There is no such thing as a folder. You have to remove files.

Folders are derived from the paths with string processing. They don't exist as far as the API is concerned. The UI does make up a fake folder for viewing. But the API does not know about it or care.

Bulk delete is an array of ALL files you want deleted. There is no delete files in this folder as there really is no folder.

[bart]

All right, got you. So I need to do the listing anyways. Will do it the DB way to fetch storage objects starting with a given path and then doing bulk delete on these objects. Thank you! I know that S3 also doesn't have a folders concept. They are just part of stored object locations. So ya, it's the same here because it's using S3 under the hood.

[GaryAustin1]

Correct. And just processing the table directly seems pretty easy versus the list command.

[bart]

Indeed it is. That is a pretty good approach. Thanks again!

[Andriiklymiuk]

well, I came up with this solution for usage in edge function (not the best one, but does the job)

function chunkArray<T>(array: T[], size: number): T[][] {
  let result: T[][] = [];
  for (let i = 0; i < array.length; i += size) {
    result.push(array.slice(i, i + size));
  }
  return result;
}

async function listAllFiles(
  client: SupabaseClient,
  bucket: string,
  path: string,
): Promise<string[]> {
  let files: string[] = [];
  const { data, error } = await client.storage.from(bucket).list(path, {
    limit: 100,
  });
  if (error) throw new Error(`Failed to list files: ${error.message}`);
  console.log("Data: " + JSON.stringify(data, null, 2));

  for (const file of data) {
    const fullPath = path + file.name;
    if (!file.name.includes(".")) {
      const subFiles = await listAllFiles(client, bucket, fullPath + "/");
      files = files.concat(subFiles);
    } else {
      files.push(fullPath);
    }
  }
  return files;
}

async function deleteFilesInChunks(
  client: SupabaseClient,
  bucket: string,
  basePath: string,
) {
  try {
    const filesToDelete = await listAllFiles(client, bucket, basePath);
    if (filesToDelete.length > 0) {
      const chunkSize = 100;
      const fileChunks = chunkArray(filesToDelete, chunkSize);
      for (const chunk of fileChunks) {
        const { data: filesRemoved, error: filesRemovedError } = await client
          .storage
          .from(bucket)
          .remove(chunk);
        if (filesRemovedError) {
          console.error("Failed to remove files:", filesRemovedError.message);
          continue;
        }
        console.log("Files removed:", JSON.stringify(filesRemoved, null, 2));
      }
    } else {
      console.log("No files to delete.");
    }
  } catch (error) {
    console.error("An error occurred:", error.message);
  }
}

deleteFilesInChunks(
  supabaseClient,
  "my-target-bucket",
  "pathToStartDeletionInMyCaseIsUserId/",
);

I know it can be heavy for many files, but has someone came up with better alternative maybe for dealing with many files deletion? or I am missing smth and supabase already has some helper?