Help with Large Zip File Generation
-
π
After having great luck on trivially small sites, I went the other way and tried it on a MUCH larger one:
~1,000 Posts and a 5.3GB Media LibraryUnfortunately, the zips that get created appear corrupt or otherwise invalid. Its size seemed appropriate, though! Around 5GB
No errors were logged during the process, such as out of memory errors of max_execution_time errors. The server has 1GB of RAM available, for reference. (Full disclosure: The was originally a memory error logged, but after actually deactivating some plugins – such as Query Monitor – and removing unnecessary ones from the Include list, that error stopped.)
Trying to unzip the result with Windows: “End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive.”
I ruled out it wasn’t a problem downloading the zip by trying to unzip it on the server as well: “cannot find zipfile directory”
Next I tried using 7zip instead, which reported “Unexpected end of data” during the process. It let me browse the result, but it was wildly incomplete.
No .zip.tmp files were present at the end, so it looked like it tried to finish the process.
I found nothing relevant in the debug logs, such as “error”, and everything seemed fine.
The WP environment has libzip 1.11.4 and zip 1.19.5, which theoretically can handle zips of this size.
I repeated the process 3 times, with the same result each time.In case it’s helpful, I’m including the tail end of the debug log (with the download links removed). I’m not sure if having multiple instances of things like “ZIP archive created:…” is to be expected or not, or if there’s anything else helpful in here. Certainly happy to share more of the log if you think it’d be helpful.
Let me know if you have any ideas, or other things to test. Happy to add additional debugging output or experiment with things in any way I can to help! Cheers
[2025-11-04 21:09:36] [class-ss-archive-creation-job.php:247] We've found our next task: create_zip_archive
[2025-11-04 21:09:36] [class-ss-archive-creation-job.php:206] Current task: create_zip_archive
[2025-11-04 21:09:36] [class-ss-archive-creation-job.php:220] Performing task: create_zip_archive
[2025-11-04 21:09:36] [class-ss-create-zip-archive.php:85] Fetching list of files to include in zip
[2025-11-04 21:09:37] [class-ss-create-zip-archive.php:94] Creating zip archive
[2025-11-04 21:09:51] [class-ss-multisite.php:265] Restored to blog: 588
[2025-11-04 21:10:51] [class-ss-multisite.php:265] Restored to blog: 588
[2025-11-04 21:10:51] [class-ss-background-process.php:422] Maybe handling: Site ID: 588
[2025-11-04 21:10:51] [class-ss-archive-creation-job.php:206] Current task: create_zip_archive
[2025-11-04 21:10:51] [class-ss-archive-creation-job.php:220] Performing task: create_zip_archive
[2025-11-04 21:10:51] [class-ss-create-zip-archive.php:85] Fetching list of files to include in zip
[2025-11-04 21:10:51] [class-ss-create-zip-archive.php:94] Creating zip archive
[2025-11-04 21:11:51] [class-ss-multisite.php:265] Restored to blog: 588
[2025-11-04 21:12:51] [class-ss-multisite.php:265] Restored to blog: 588
[2025-11-04 21:12:51] [class-ss-background-process.php:422] Maybe handling: Site ID: 588
[2025-11-04 21:12:51] [class-ss-archive-creation-job.php:206] Current task: create_zip_archive
[2025-11-04 21:12:51] [class-ss-archive-creation-job.php:220] Performing task: create_zip_archive
[2025-11-04 21:12:51] [class-ss-create-zip-archive.php:85] Fetching list of files to include in zip
[2025-11-04 21:12:51] [class-ss-create-zip-archive.php:94] Creating zip archive
[2025-11-04 21:13:28] [class-ss-task.php:44] Status message: [create_zip_archive] ZIP archive created:
[2025-11-04 21:13:28] [class-ss-archive-creation-job.php:223] Task performed: 1
[2025-11-04 21:13:28] [class-ss-archive-creation-job.php:247] We've found our next task: wrapup
[2025-11-04 21:13:28] [class-ss-background-process.php:422] Maybe handling: Site ID: 588
[2025-11-04 21:13:28] [class-ss-archive-creation-job.php:206] Current task: wrapup
[2025-11-04 21:13:28] [class-ss-archive-creation-job.php:220] Performing task: wrapup
[2025-11-04 21:13:28] [class-ss-wrapup-task.php:23] Deleting temporary files
[2025-11-04 21:13:28] [class-ss-task.php:44] Status message: [wrapup] Wrapping up
[2025-11-04 21:13:28] [class-ss-archive-creation-job.php:223] Task performed: 1
[2025-11-04 21:13:28] [class-ss-archive-creation-job.php:242] This task is done and there are no more tasks, time to complete the job
[2025-11-04 21:13:28] [class-ss-archive-creation-job.php:287] Completing the job
[2025-11-04 21:13:28] [class-ss-archive-creation-job.php:427] Status message: [done] Done! Finished in 00:24:40
[2025-11-04 21:13:51] [class-ss-multisite.php:265] Restored to blog: 588
[2025-11-04 21:13:51] [class-ss-multisite.php:265] Restored to blog: 588
[2025-11-04 21:14:41] [class-ss-task.php:44] Status message: [create_zip_archive] ZIP archive created:
[2025-11-04 21:14:42] [class-ss-archive-creation-job.php:223] Task performed: 1
[2025-11-04 21:14:42] [class-ss-archive-creation-job.php:247] We've found our next task: wrapup
[2025-11-04 21:14:42] [class-ss-archive-creation-job.php:287] Completing the job
[2025-11-04 21:14:42] [class-ss-archive-creation-job.php:427] Status message: [done] Done! Finished in 00:25:54
[2025-11-04 21:16:40] [class-ss-task.php:44] Status message: [create_zip_archive] ZIP archive created:
[2025-11-04 21:16:40] [class-ss-archive-creation-job.php:223] Task performed: 1
[2025-11-04 21:16:40] [class-ss-archive-creation-job.php:247] We've found our next task: wrapup
[2025-11-04 21:16:40] [class-ss-archive-creation-job.php:287] Completing the job
[2025-11-04 21:16:40] [class-ss-archive-creation-job.php:427] Status message: [done] Done! Finished in 00:27:52
You must be logged in to reply to this topic.