Supporting ColdFusion with Command Line Programs

Here are some supporting Windows command line programs that I've been using with Adobe ColdFusion. (I'll be posting another list of "supporting java libraries" soon.)

7-Zip (GNU LGPL)

https://www.7-zip.org/

7-Zip is a file archiver with a high compression ratio for ZIP and GZIP formats, which is between 2 to 10% better than its peers, depending on the exact data tested. And 7-Zip boosts its very own 7z archive format that also offers a significantly higher compression ratio than its peers—up to 40% higher! Supports packing and unpacking 7z, XZ, BZIP2, GZIP, TAR, ZIP and WIM formats. 7za.exe supports only 7z, xz, lzma, cab, zip, gzip, bzip2 and tar formats Command Line Version User's Guide

NOTE: We started using this before ColdFusion 8 added the feature. It's still faster and can function in the background without a cfthread of fear of timing out.

REM Zip all files & delete after compression
C:\7-zip\7za.exe a -bd -sdel c:\allFiles.zip c:\DirToZip\*.*

BLAT (Public Domain)

http://www.blat.net/

Blat is a Public Domain console utility that sends the contents of a file in an email message using the SMTP protocol.

NOTE: We occasionally use BLAT with BAT files to provide an email notification when a task has completed.

REM Send email message
C:\blat\blat.exe -body "Bat file executed" -subject "Files downloaded" -to user@test.com -f fromAddress@test.com -server smtp.text.com:25

Bulk Rename Command (free)

https://www.bulkrenameutility.co.uk/

Bulk Rename Command is the free command-line version of Bulk Rename Utility. With Bulk Rename Command (BRC) you can:

• Create scheduled jobs using the Windows Scheduler.
• Perform specific tasks repetitively without user intervention.

NOTE: This EXE is very fast. The alternative is to recursively use CFDirectory, loop and use CFile to rename files and hope it doesn't time-out.

REM Remove "ML" prefix from all images within a directory.
c:\brc\BRC64.exe /DIR:"D:\Photos" /QUIET /NOFOLDERS /PATTERN:"ML*.jpg" /REMOVECHARS:ML /EXECUTE

CURL (Open Source)

https://curl.haxx.se/

Supports DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, HTTP/2, cookies, user+password authentication (Basic, Plain, Digest, CRAM-MD5, NTLM, Negotiate and Kerberos), file transfer resume, proxy tunneling and more.

NOTE: We started using CURL because many third-party services provide sample code. We also ran into an RETS issue where CFHTTP basic authentication wouldn't reauthenticate (which RETS may do according to their spec). We also use CURL to perform large, bulk download operations in the background without having to worry about cfthreads or time-outs. Want a CFC? Check out cf-curl.

REM Perform POST to API Endpoint w/custom user agent
c:\CURL\CURL.exe -L -A "CustomUserAgent" https://api.test.com/myEndPoint -d "ID=0FA7AE06-7E81-442D-B13F-F96F7315FF6A"

ExifTool (free)

https://exiftool.org/

ExifTool is a platform-independent Perl library plus a command-line application for reading, writing and editing meta information in a wide variety of files. ExifTool supports many different metadata formats including EXIF, GPS, IPTC, XMP, JFIF, GeoTIFF, ICC Profile, Photoshop IRB, FlashPix, AFCP and ID3, as well as the maker notes of many digital cameras by Canon, Casio, DJI, FLIR, FujiFilm, GE, GoPro, HP, JVC/Victor, Kodak, Leaf, Minolta/Konica-Minolta, Motorola, Nikon, Nintendo, Olympus/Epson, Panasonic/Leica, Pentax/Asahi, Phase One, Reconyx, Ricoh, Samsung, Sanyo, Sigma/Foveon and Sony.

NOTE: Great utility for retrieving keywords, caption, description and headline from JPGs.

REM Get EXIF data in JSON format
C:\exiftool\exiftool.exe -a -G1 -json -scanForXMP -b -L -m -c "%.6f %.8f" -d "%d/%m/%Y %H:%M:%S" c:\image.jpg

Exiv2 (Open Source)

https://www.exiv2.org/

Exiv2 is a Cross-platform C++ library and a command line utility to manage image metadata. It provides fast and easy read and write access to the Exif, IPTC and XMP metadata and the ICC Profile embedded within digital images in various formats.

NOTE: 2-4 times faster than ImageGetEXIFMetaData.

REM Parse colon-delimited data
C:\Exiv2\Exiv2.exe c:\image.jpg

REM Parse space-delimited values. Generate struct using key dot-delimited keys.
C:\Exiv2\Exiv2.exe -Pnt c:\image.jpg

FastCopy (freeware with no guarantee)

https://fastcopy.jp/en/

• FastCopy is the Fastest Copy/Backup Software on Windows
• It supports UNICODE and over MAX_PATH (260 characters) file pathnames.
• Because it uses multi-threads for Read/Write/Verify, Overlapped I/O, Direct I/O, so it brings out the best speed of devices.
• It supports Include/Exclude filter like a UNIX wildcard.
• It runs fast and does not hog resources, because MFC is not used. (Designed using Win32 API and C Runtime only)

NOTE: We started using this to copy mailbox files from the mail server rather than using CFPop or CFFile. This is a much better solution than using a cfthread and risking a timeout.

REM Copy (Overwrite) - Always copy/overwrite all.
c:\FastCopy\FastCopy.exe /cmd=copy /no_ui "\\SERVER1\c\logs\" /to="c:\logs\"

REM Sync (Size/date) - Copy if size or date is different, or doesn't exist. In addition, delete all destination files/directories that don't exist in source.
c:\FastCopy\FastCopy.exe /cmd=Sync /no_ui "\\SERVER1\c\logs\" /to="c:\logs\"

REM Move (Overwrite) - Always copy/overwrite all and delete all copied source files. (It is able to change "Move (Overwrite all)" -> "Move (Size/date)" in settings dialog.)
c:\FastCopy\FastCopy.exe /cmd=move /no_ui "\\SERVER1\c\log\" /to="c:\logs\"

File Locator Pro (Commercial) / Agent Ransack (Free)

https://www.mythicsoft.com/filelocatorpro/

FileLocator Pro's start up values can be altered through information passed in the command line. If an output file is specified then the actual User Interface is not displayed and the results are streamed directly to the file. Alternatively, there is also a console version of the program which displays information on the console instead of through the Windows user interface. -Command Line Help

NOTE: We use this to scan our project directories for regex patterns to identify deprecated code that we plan to update. It generates a CSV file that identifies the file, line number and matched string.

REM Search multiple hosts/paths for CFINPUT; save results to CSV file.
C:\FileLocatorPro\flpsearch.exe -c ^<CFINPUT -cee -s -d \\SERVER1\D\www\;\\SERVER2\D\www\ -f *.cfm;*.cfc -fed -ofc -o D:\FileLocatorPro\results_CFINPUT.csv

FileList

https://www.jam-software.com/filelist

FileList is a command line utility providing a list of files of the selected path in the CSV format. The list includes the file name, the size, the file path, the last access and change date, date of creation etc. You can easily import all results to a spreadsheet or database. (Use Cases and Online Manual)

NOTE: This is a lot faster than recursively using CFDirectory to retrieve a directory tree.

c:\FileList\FileList.exe /MINDATE 2018-11-01 /NOTITLE /FULLPATH /FILTER *.cfm  /COLUMNS MD5 \\SERVER1\D\www\ > c:\FileList\SERVER1_ChangedFiles.csv

REM List of all photos and also shows their size in pixels.
c:\FileList\FileList.exe /INCLUDE *.jpg /USECOLUMNS FullPath,Image.Dimensions "c:\MyMobilePhone\DCIM\Camera"

GhostScript (GNU Affero GPL license)

https://www.ghostscript.com/

Ghostscript is a high-performance Postscript and PDF interpreter and rendering engine with the most comprehensive set of page description languages (PDL's) on the market today and technology conversion capabilities covering PDF, PostScript, PCL and XPS languages.

NOTE: Sometimes client-provided or Adobe-generated PDFs can be quite large. GhostScript can greatly reduce the filesize.

REM Reduce bloated PDF filesize.
c:\gs\gswin64.exe -sDEVICE=pdfwrite -dNOPAUSE -dQUIET -dBATCH -sOutputFile=C:\gs\out.pdf C:\gs\in.pdf

GraphicsMagick (MIT)

http://www.graphicsmagick.org/

GraphicsMagick is the swiss army knife of image processing providing a robust and efficient collection of tools and libraries which support reading, writing, and manipulating an image in over 89 major formats including important formats like DPX, GIF, JPEG, JPEG-2000, PNG, PDF, PNM, TIFF, and WebP.

NOTE: Faster than CFImage and can bulk optimize an entire sub-directory of images. We started using this because some iPhone 4 JPGs were not able to be read by CFImage.

REM Aspect Crop Demo
c:\graphicsmagick\Convert.exe -colorspace rgb -strip -filter Lanczos -auto-orient -density 72x72 -size 350x350 "c:\in\largeimage.png" -thumbnail "350x350^" -gravity Center -extent 350x350 -interlace Line -quality 75 +profile "*" "c:\out\AspectCrop.png"

REM ResizeFit Demo
c:\graphicsmagick\Convert.exe -colorspace rgb -strip -filter Lanczos -auto-orient -density 72x72 "c:\in\largeimage.png" -resize "250x250>" -interlace Line -quality 75 +profile "*" "c:\out\ResizeFit.png"

REM Montage (combine images and auto-resize)
c:\graphicsmagick\montage.exe -geometry 120x120 -tile 4x3 -bordercolor black c:\image1.png c:\image2.png c:\image3.png c:\image4.png c:\image5.png c:\image6.png c:\image7.png c:\image8.png

REM Watermark
c:\graphicsmagick\composite.exe -gravity southeast -geometry +20+10 c:\Watermark.png c:\largeimage.png

REM Generate animated GIF (auto-resize)
c:\graphicsmagick\convert.exe -thumbnail 200x200 -background "#fff" -gravity center -extent 200x200 -delay 75 -colorspace rgb -colors 32 -dither -dispose 1 -loop 0 C:\image1.png c:\image2.jpg c:\image3.jpg c:\image4.png c:\image5.png c:\animated.gif

REM Contact sheet
c:\graphicsmagick\montage.exe +adjoin -compress JPEG +frame +shadow -font Arial -pointsize 15 -tile 4x1 -label %t -geometry 350x350+4+8> -title "Sample Contact Sheet" c:\image1.png c:\image2.jpg c:\image4.jpg c:\image5.png c:\contactsheet.png

REM Trim background
c:\graphicsmagick\convert.exe c:\image.png -fuzz 50% -trim +repage c:\image_trimmed.jpg

HTTrack (free GPL)

https://www.httrack.com/

It allows you to download a World Wide Web site from the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from the server to your computer. HTTrack arranges the original site's relative link-structure. Simply open a page of the "mirrored" website in your browser, and you can browse the site from link to link, as if you were viewing it online. HTTrack can also update an existing mirrored site, and resume interrupted downloads. HTTrack is fully configurable, and has an integrated help system. Users Guide

NOTE: We use HTTrack to manually create offline copies of websites. The command line has enabled us to provide a one-click export that allows clients to create a local static version of their website for use at tradeshows that may not offer internet access.

REM Create local, browseable copy of a website
c:\httrack\httrack.exe "https://www.mywebsite.com/" -O "c:\mywebsitelocal" "+*.mywebsite.com/*" -v

Kid3 - Audio Tagger (Open Source)

https://kid3.kde.org/

If you want to easily tag multiple MP3, Ogg/Vorbis, FLAC, MPC, MP4/AAC, MP2, Opus, Speex, TrueAudio, WavPack, WMA, WAV and AIFF files (e.g. full albums) without typing the same information again and again and have control over both ID3v1 and ID3v2 tags, then Kid3 is the program you are looking for.

NOTE: I don't believe that Adobe has any built-in support for managing media metadata. We host a couple of radio shows and Kid3 has enabled us to commercially brand all media with unique artwork and the ID3 data that we required.

REM Embed ID3 data in an MP3
c:\kid3\kid3-cli.exe -c "set title 'My Track Title'" -c "set artist 'My Artist'" -c "set album 'My Album'" -c "set track '2'" -c "set composer 'My Composer Name'" -c "set 'album artist' 'My Artist Name'" -c "set 'disc number' 90" -c "set date '2020-04-09T12:47:01'" -c "set language 'eng'" -c "set genre 'Speech'" -c "set copyright 'My Copyright®'" -c "set WXXX https://www.mywebsite.com/" -c "set comment 'Visit MyWebsite® to listen to a recent podcast.'" -c "set picture:'c:\test\MyCoverArt.jpg' 'Cover (front)'" c:\test\myAudio.mp3

Markdown Monster (Commercial)

https://markdownmonster.west-wind.com/

Markdown Monster is a Markdown editor and viewer that lets you edit Markdown with syntax highlighting and fast text entry. A collapsible, synced, live preview lets you see your output as you type. Easily embed images, links, emojis and code using Markdown text or use our gentle UI helpers that simplify many operations with a couple of keystrokes. Paste images from the clipboard or drag and drop from Explorer or our built-in folder browser. Inline spell-checking and word counts keep your content streamlined unobtrusively. Read CLI Commands

NOTE: Very good Markdown editor. (It's what I use.) While discussing using ColdFusion to parse Markdown as HTML, I stated that I was impressed with whatever MM was doing and the author reached out and added command line support. How cool is that! (NOTE: The MM author also wrote "West Wind DeleteFiles".)

c:\markdownmonster\mmcli.exe markdowntohtml -i "c:\test.md" -o "c:\test.html" --rendermode fragment

MediaInfo (Open Source)

https://mediaarea.net/en/MediaInfo

MediaInfo is a convenient unified display of the most relevant technical and tag data for video and audio files.

• Export information as text, CSV, HTML...

REM Get Basic info (parse for YCCB|CMYK to determine if CMYK)
c:\mediainfo\mediainfo.exe --Language=raw --ReadByHuman=0 "--Output=Image;%ColorSpace%" --Legacy=0 c:\image.jpg

REM Parse response to identify headers & colon-delimited keys & values
c:\mediainfo\mediainfo.exe --Language=raw -Full --ReadByHuman=1 c:\image.jpg

Pandoc

https://pandoc.org/

If you need to convert files from one markup format into another, pandoc is your swiss-army knife. Pandoc can convert between the following formats: Markdown, ASCIIDoc, Textile, (X)HTML4, HTML5, EPUB, LaTeX, DOCX, ODT, CSV and more.

REM Convert Markdown to HTML
c:\pandoc\pandoc.exe c:\test1.md -f markdown -t html -s -o c:\test1.html

REM Convert HTML to Markdown
c:\pandoc\pandoc.exe c:\test1.html -f html -t markdown -o c:\test1.md

PDFtk (free/commercial)

https://www.pdflabs.com/tools/pdftk-the-pdf-toolkit/

PDFtk Server can:

• Merge PDF Documents or Collate PDF Page Scans
• Split PDF Pages into a New Document
• Rotate PDF Documents or Pages
• Decrypt Input as Necessary (Password Required)
• Encrypt Output as Desired
• Fill PDF Forms with X/FDF Data and/or Flatten Forms
• Generate FDF Data Stencils from PDF Forms
• Apply a Background Watermark or a Foreground Stamp
• Report PDF Metrics, Bookmarks and Metadata
• Add/Update PDF Bookmarks or Metadata
• Attach Files to PDF Pages or the PDF Document
• Unpack PDF Attachments
• Burst a PDF Document into Single Pages
• Uncompress and Re-Compress Page Streams
• Repair Corrupted PDF (Where Possible)

PDFtk Server does not require Adobe Acrobat or Reader, and it runs on Windows, Mac OS X and Linux.

NOTE: We've had to use this because sometimes ColdFusion refuses to work with 3rd-party generated PDF files.

REM Concat Multiple PDFs into a single PDF
c:\PDFtk\bin\pdftk.exe PDF1.pdf PDF2.pdf cat output final.pdf flatten compress dont_ask

ScriptFTP (Commercial)

https://www.scriptftp.com/

ScriptFTP is an FTP client designed to automate file transfers. It follows the commands written on a text file (also called script file) and makes the uploads or downloads automatically.

NOTE: An MLS client required an XML file and many JPGs to be uploaded to a their SFTP server. We used ColdFusion to generate the XML & a BAT file to automate the image resizing & renaming according to their spec (using GraphicsMagick) and ScriptFTP performed the massive file upload.

REM # Connect to ftp.myhost.com as myuser
OPENHOST("ftp.myhost.com","myuser","123456")
REM # Upload the file notes.txt using its full path
PUTFILE("C:\My Documents\notes.txt")
REM # Change current local directory to C:\Users\Eli\Documents
LOCALCHDIR("C:\Users\Eli\Documents")
REM # Upload the file sales.xls located in the current local directory
PUTFILE("sales.xls")
REM # Upload the file backup.zip located in the current local directory under the folder backups.
PUTFILE("backups\backup.zip")
REM # Change current remote directory to remote_images
CHDIR("remote_images")
REM # Upload every jpg file in the subdirectory images to the remote directory remote_images
PUTFILE("images\*.jpg")
REM # Close the connection
CLOSEHOST

S3Express (Commercial)

https://www.s3express.com/

S3Express is a command line software utility for Windows. Easily upload, query, backup files and folders to Amazon S3 storage, based upon multiple flexible criteria. Create custom batch scripts, list Amazon S3 files or entire folders, filter them with conditions, query, change object metadata and ACLs. Quickly upload only new or changed file using multipart uploads and concurrent threads. (More info)

NOTE: Here are some performance comparisons.

• Adobe ColdFusion 10: 110 seconds
• AmazonS3 CLI: 40 seconds
• S3Express: 16 seconds!

REM Upload all JPGs in a folder using 16 threads
c:\s3express\s3express.exe put c:\folder\*.jpg mybucket -s -t:16 -optimize

REM Upload 27mb file
c:\S3Express\S3Express.exe "loadauth NAME" "put c:\file.zip s3dir/s3subdir/ -t:10 -rr -minoutput -optimize -mul:5 -nomulmd5 -nomd5existcheck -nobucketlisting -move -tofile:c:\filelog.txt" -exit

REM One-way-copy+delete files/directories from a local path to a remote S3 server based on MD5 hash. "_gsdata_" directory is exluded. To work with BAT file, a separate TXT file and the "exec" command will need to be used due to brackets in the commandline.
c:\S3Express\S3Express.exe -ini:OneWaySync.txt.txt -nm

REM Contents of "OneWaySync.txt"
put c:\projectscripts\ projectscripts/ -s -onlydiff -t:3 -purge -purgeabort:NEVER -exclude:_gsdata_ -tofile:c:\projectlogs\s3express_<*date*><*hour*><*minute*><*second*>.txt

West Wind DeleteFiles (MIT)

https://github.com/RickStrahl/DeleteFiles

DeleteFiles is useful for common directory cleanup tasks, such as cleaning out temp folders, or log or backup files and so on. The timed features are especially useful for clearing out log and backup directories of dated files that are older than a specified number of days.

DeleteFiles is also useful for deleting deeply nested folder hierarchies that can't be deleted in Windows Explorer or other standard file utilities due to path lengths exceeding Windows default limitations (MAX_PATH). DeleteFiles can easily delete long paths as well as read-only, system and hidden files.

Because DeleteFiles is a small self-contained, single Exe it's easy to install and access it and use it for admin tasks in server batch/script files. Personally I use a number of DeleteFiles operations of scheduled tasks to manage database log cleanup (toss all logs older than 10 days), clean out all system temp folders, and scour the entire system for known temp files.

NOTE: This EXE is so fast. The alternative is to recursively use CFDirectory, loop and use CFile to delete files & directories and hope it doesn't time-out.

REM deletes all files in temp folder recursively & deletes empty folders
DeleteFiles c:\temp\*.* -r -f

REM delete .tmp files recursively from current folder down
DeleteFiles *.tmp -r

REM deletes all files leaves folders
DeleteFiles ""c:\My Files\*.*"" -r

REM delete files 10 days or older
DeleteFiles c:\temp\*.* -r -f -d10

REM delete files older than an hour
DeleteFiles c:\temp\*.* -r -f -s3600

REM delete thumbs.db on entire drive
DeleteFiles c:\thumbs.db -r

WGET (Open Source)

https://www.gnu.org/software/wget/

GNU Wget is a free software package for retrieving files using HTTP, HTTPS, FTP and FTPS the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.

NOTE: I've used WGET in multiple scenarios; 1) to generate a MAFF file (no longer supported) and 2) request a URL and then sequentially follow all same-site links.

REM Recursively & sequentially spider target URL and all links w/1 second delay
c:\wget\wget.exe --spider --recursive --wait=1 --execute robots=off --user-agent="CustomUserAgent" "https://www.mysite.org/SpecialScript.cfm"

WKHTMLTOPDF (Open Source)

https://wkhtmltopdf.org/

wkhtmltopdf and wkhtmltoimage are open source (LGPLv3) command line tools to render HTML into PDF and various image formats using the Qt WebKit rendering engine. These run entirely "headless" and do not require a display or display service. Check out the auto-generated wkhtmltopdf manual.

NOTE: Faster than CFDocument and better compliance with CSS3 & HTML5 standards. PDF filesize is less bloated. Supports headers & footers, absolute positioning, opacity, webfonts, SVG images, JS DOM manipulations (embedded Google Map, JS QRCode), 1px borders, TOC & headers/footers. Here's a basic ColdFusion CFTag that I wrote in 2015.

REM Convert a webpage to a PDF
c:\wkhtmltopdf\wkhtmltopdf.exe https://www.google.com c:\google.pdf

Zint (Open Source)

http://www.zint.org.uk/

Zint is a suite of programs to allow easy encoding of data in any of the
wide range of public domain barcode standards and to allow integration of
this capability into your own programs. Output can be saved as BMP, EPS, GIF, PCX, TIF, EMF, PNG or SVG.

NOTE: Generates 93 different bar code formats. Many java libraries I've used has been dedicated to a handful of formats and have very limited options.

REM Generate royal blue QR code using SVG format (looks great w/WKHTMLTOPDF PDFs)
C:\zint\zint.exe -o "c:\qrcode.svg" -b 58 --fg=002366 -d ""Data to encode""