ziplim.bat to create .zips based on input filesize
[page last modified 2019-02-10]

This is a Windows command script (.bat) to create standalone, numbered .zip files from a set of input files.  A new output .zip is started when the sum of input filesizes hits a user-specified megabyte limit.  The "7z.exe" tool is used to create the output .zips and must be in the path (google and download if you don't have it).

This isn't a particularly useful tool; what caught my interest is to see if it could be accomplished -- handling (in a pure .bat file) input file specifications with wildcards and also input filenames containing special characters.   Also there is no filesize math limit (hundred gigabyte input/output files will work).  I believe the result .bat accomplishes the goal, though the code is painful (and it's slow to deal with large numbers of input files).

The first argument is a decimal number which holds the number of input megabytes before a new output .zip is started; e.g. 100 will group each 100 megabytes worth of input files into a separate .zip file.  There are two special cases: (1) -1 means "infinite" input size and will create a single output .zip; (2) 0 means create a separate .zip output for every input file.

Following the first argument you may put 7z options separated by spaces.   Each option must be prefixed by a hyphen.  An example might be -mx0 to disable compression of the input, or -t7z to output .7z files instead of .zip.   These don't affect the .bat logic; they are simply collected and passed to the underlying 7z.exe tool when it is invoked.

The second argument is the path and prefix used for all output file names.   Appended to each ouput file name will be 001, 002, etc (if the first argument megabyte limit is -1 no number is appended).  A special testing case is "nul" -- the input file sizes are totaled but no output is generated.

Finally zero or more input filespecs may be present.  If no filespecs are present the contents of the current and all sub-directories is assumed.   Example filespecs:  (1) "file 1.txt"  (2) file*.txt

Some example command lines with descriptions:

   ziplim -1 foo
      Compress all files in the current directory and below into foo.zip
      (equivalent to  7z a -tzip foo).

   ziplim 0 out foo*.* bar*.*
      Compress all files in current directory whose names begin with foo or bar into files
      named out001.zip, out001.zip, etc.  There will be one input file per output file.

   ziplim 100000 -mx0 zips *.zip
      Group .zip files into 100 gig output files, without recompression.
The final example is what started me on this project... I had a huge number of .zip files that I wanted to collect into larger files of around 100 gig each for uploading.


Change Log

2/10/2019  Original release.

Click here to download  ziplim190210.zip


Widget is loading comments...

You are visitor 4413       Go to Home Page


@echo off& setlocal disabledelayedexpansion set pn=%~n0& set tf1=&set tf2=& if "%~1"=="" goto help set r=%random%& rem set temp file names; test we can write to temp dir set tf1="%tmp%\%pn%-%r%a.tmp"& set tf2="%tmp%\%pn%-%r%b.tmp" type nul>%tf1%& if not exist %tf1% echo %pn%: *** %tf1% error& exit/b set/a ofc=0,ml=-1& set/a ml=%~1 2>nul if not "%ml%"=="%~1" echo %pn%: *** bad megabyte limit: %~1& goto exit if %ml% geq 0 set ofc=1 rem set zo= 7z options (if any). arg of -- will stop option gathering. set "zo=-tzip -spf2"& set x=zip :oz set "a=x%~2" if /i "%a:~1,2%"=="-t" set zo=%zo:-tzip=%& set x=%a:~3% if "%a:~1,1%"=="-" set zo=%zo% %2& shift/2& if not "%a%"=="x--" goto oz rem insure output file(s) don't exist set a="%~2*.%x%"& if "%~2"=="" goto help if exist %a% echo %pn%: *** output file(s) %a% exist& goto exit rem collect input filenames (and contents of @filelists) into %tf1% set/p ="%pn%: collecting input info... "%tf1%& goto inf_done :inf set "a=x%~3"& set t=x%~a3 if "%a:~1,1%"=="@" ( if "%a:~2%"=="" (find/v "") else find/v ""<"%a:~2%")>>%tf1%& goto inx set w=& if x%t:~1,1%==xd if "%a%%a%"=="%a:?=%%a:**=%" set w=\*.* for %%g in ("%~3%w%") do echo %%g>>%tf1% :inx if not "%~4"=="" shift/3& goto inf :inf_done rem count and validate input files exist and sum input file sizes. set/a ifc=tm=tmr=e=0,MB=1000000& ver>nul& rem ver 0's errorlevel for /f "usebackq delims=" %%f in (%tf1%) do ( set/a a=%%~zf 2>nul& if errorlevel 1 set "f=%%~f"& call :vif_spec %%~zf set/a ifc+=1,tmr+=a%%MB,tm+=a/MB+tmr/MB,tmr%%=MB ) if %e%==1 (goto exit) else goto vif_spec_skip rem special cases: 1. file not found; 2. file size geq 2**31 :vif_spec setlocal enabledelayedexpansion& set "a=%1" if x%1==x (if %e%==0 echo.& set e=1)& echo %pn%: *** not found: !f! if not x%1==x set/a tm+=!a:~0,-6!& set a=1!a:~-6!-MB endlocal& set/a a=%a%+0,e=%e%,tm=%tm%& ver>nul& exit/b :vif_spec_skip rem output count of files and total bytes set/a a=tmr,f=tmr+MB if %tm% geq 1 set a=%tm%%f:~1% call :nfmt f %ifc% a %a% echo %f% files %a% bytes echo .>>%tf1%& if %ifc%==0 goto exit rem create output (note: %tf1% file list is terminated with a .) if /i "%~2"=="nul" goto exit setlocal enabledelayedexpansion set/a m=mr=pm=pmr=p=np=0& if %tm%==0 set p=100 set/p ="%pn%: creating output... "nul& for /f "usebackq delims=" %%f in (%tf1%) do ( if "%%f"=="." (set ml=0) else ( set s=%%~zf& set r=%MB%!s! set/a mr+=1!r:~-6!-MB,m+=!s:~0,-6!+mr/MB,mr%%=MB setlocal disabledelayedexpansion& echo %%f>>%tf2%& endlocal ) if !ml! geq 0 if !m! geq !ml! if exist %tf2% ( set fx=&if !ofc! geq 1 set/a fx=ofc,ofc+=1& if !fx! leq 99 ( set fx=00!fx!& set fx=!fx:~-3!) 7z a %zo% "%~2!fx!" @%tf2%>nul if errorlevel 1 echo.& echo %pn%: *** 7z error& goto exit set/a pmr+=mr,pm+=m+pmr/MB,pmr%%=MB,m=mr=0 if !p! lss 100 set/a p=pm*100/tm& if !p! geq !np! ( set/p ="!p!%% "nul ) ) echo.& goto exit rem Routine to format commas into decimal number (up to 21 digits). rem %1= output variable name, %2= number to format (leading - allowed) :nfmt setlocal enabledelayedexpansion& set n=%2 for /l %%i in (-3,-4,-23) do ( if not "!n!"=="!n:~%%i!" if not "!n:~0,%%i!"=="-" ( set n=!n:~0,%%i!,!n:~%%i!) ) endlocal& set %1=%n% if not "%~3"=="" (shift& shift& goto nfmt) else exit/b :help echo. echo Compresses files into archives. PaulHoule.com 2/10/2019. echo. echo nnn Decimal input size limit in megabytes (-1 for no limit). echo When input limit reached a new output archive is started. echo {-opts} Arg(s) here prefixed with hyphen are used as 7zip echo options (for example -mx0 for no compression). echo output Output path and filename -- output files are given echo this name postfixed with 001, 002, etc. echo {inf} Input filespecs (eg: *.exe readme.txt) -- if none echo uses files in current and sub-directories. echo @file uses file as filelist (@ alone uses stdin). echo.& echo Example 1: create 1 gig archives gig{nnn} w/o compression echo %pn% 1000 -mx0 gig *.* :exit del %tf1% %tf2% >nul 2>nul