Syncaid is a powerful, easily-configurable Internet downloader tool, designed to be reusable, dynamic and very portable. Oh yeah, and free! Current version is 18.104.22.168.
Without a settings file, the application itself (weighing in at less than 200kb) is quite portable - but it doesn't do much without one. Running the program without parameters and without a settings file of the same name beside it will open a minimal configuration window, which provides you the ability to create a custom settings file easily.
Syncaid supports more than a dozen different settings for each Sync configuration.
On your PC
On the Web
Sample Configuration Files
On your PC
Configuring a Syncaid settings file is intended to be both feature-complete as well as user-friendly. Running the application without a settings file enables you to create a simple settings file within a form.
By default all options are disabled, and must be enabled in order to have an effect. Also, the Sync group that is loaded by default is the one with the same name as the configuration file. Thus,will load the [mysettings] Sync options. Sync files, Sync groups, and Sync Options are case-insensitive, though the URLs processed may be case-sensitive, depending on the server you are connecting to.
The source option
While no settings are required, Syncaid will not attempt to download anything at all if there is no source or includes. The source option has two different implementations. Used in conjunction with the direct option, it can be used to address a single file for downloading. Without the direct option enabled, Syncaid assumes the URL is to a web page, PAD file, or text file containing links to the resources you wish to download.
The direct option
The simplest Sync file would include only 3 lines, as so:
This would cause Syncaid to downloadand store it in the folder beside Syncaid.
Resource Selection & Filters
The after and before options
Most pages will require a little bit of massage to get exactly what you want out of them. If a page has a consistent structure, you can use before and after to extract a specific region of the resulting html or text for parsing. Think of it as a way to eliminate all but the region of text where the important links are.
Important note! When both before and after have tokens assigned to them, the after token is parsed before the token assigned to before!
If a token is not found, any limitations it might have imposed are ignored. Thus, if the alphabet (a to z) were the value to be parsed, with the following settings, r to z would be returned:
Again using the alphabet for demonstration purposes, the following would return everything between (but excluding) j and s:
Also note that carriage returns cannot be used in before and after tokens.
The saveas option
If you are not happy with the filename that is provided by the site, parsed automatically or just want to be creative, you can use the saveas option to name a downloaded anything you wish. This option works only when the direct option is enabled.
The extensions option
The original purpose of Syncaid is to act as a synchronization tool, obtaining the most current versions of applications and installers. With that in mind, you can see how important a simple option to get only files with a certain extension could be. This option can have multiple extensions, separated by a pipe character (|) or a comma (,).
One of the limitations of the extensions option is simply that the files you wish to match will not always be linked with the correct extension, so parsing the source cannot provide every instance of those files. In these instances, your next best option is match.
Note that type is a functional alias of extensions. Either will work, but if both are included, extensions is the preferred value.
The match option
The match option is for those situations when you need finer control over exactly what is downloaded. With this option enabled, Syncaid parses the URLs from the source, first applying before and after to limit the context range, then it tests the resulting URL code against the match regular expression pattern.
The distinction here is that in some instances only a partial URL is returned to the client to direct to a resource. For example, when a link includes only the filename:
In the above instance, the text that is tested for a regular expression match is only. While in the case of the following link the entire URL would be tested for regular expression matching:
The last option
A common requirement on sites that provide many mirrors or a series of versioned files is the need to exclude all but a single link from the resulting series. The easiest method of obtaining only the last item in one of these series is the last option, which parses the page for links matching the other requirements, but then excludes all but the last match.
The limit option
In some situations you'll want to obtain more than the first or last matching result from a set. The limit option, used in conjunction with either first or last enables you to sync that many files from the beginning or end of the matching results series.
This example would download the last three executable files:
The first option
A common requirement on sites that provide many mirrors is the need to exclude all but a single link from the resulting series. The easiest method of doing so is the first option, which parses the page for links matching the other requirements, but then excludes all but the first match.
The child option
Because of Syncaid's recursive and reiterative power, you'll find that you can chain configuration files to perform two, three or even several hundred operations in sequence. While that provides you amazing options for downloading "many" files, it can easily overwhelm the originating folder (the folder containing Syncaid), with the volume of files downloaded.
To cope with this issue, Syncaid provides the child option to store files within a child directory (subdirectory) from where the Syncaid application resides.
Assuming Syncaid is running from, the following setting would download matching files to .
This option would store matching files within:
Troubleshooting and Information
The browser option
Some websites are very elite. They provide only certain types of browsers access to their content, so it's important that you're able to act like you're using one of those browsers. The default Syncaid User-Agent string identifies it as IE6 on Windows XP. You can use the browser option to change that to anything you want.
The debug option
If something isn't working right, you can enable the debug option in order to have the configuration file report important troubleshooting details. Open the configuration file after or during processing to see the history data.
The notes option
Have you ever gone looking for something, only to find that it's already in your hand? That's not my favorite type of experience, so I try to document what things do as I initially create it. That makes it possible to actually look back at my notes and see where my head was at the time. If a website changes their structure, filenames, or content, it may become necessary to revise the Sync configuration file.
You can take advantage of the notes field to document your Sync file, or you can manually edit your Sync files to add comments to it. Start any line with a semicolon to make a comment.
; ...and so is this.
Yeah, I know. Most of Syncaid is advanced...but the following options are really advanced. Use them at your own risk.
Syncaid is designed for portability and utility. Clearly, the most portable application is one that doesn't even require a configuration to follow it around. The Syncaid clipboard parsing engine is designed specifically for those of us that might need a custom configuration, or need to share it with people that aren't tech-savvy enough to save a Synfig file.
Create the content the same way, with the only caveat being that you can't use a root section based on the file name. Since there isn't a file name, that doesn't work. You can get around this limitation by using includes option.
The extract option
If you have 7-zip installed and in your path environment variable, enabling this option for a Sync configuration will attempt to extract all compatible compressed files to the same directories where the resource was downloaded. For example, take the following Sync configuration:
This will extract the contents ofto the folder Syncaid resides within or as designated by the child setting. It will also keep a copy of the file within that folder, as a precaution in case 7-Zip is either not installed, or not within the path.
Now take this configuration:
This would extract the contents of all of the four types of compressed files found on the list atto the folder . Pretty nifty, eh?
The execute option
I'll take a moment to remind you - use these advanced options at your own risk!
The execute option enables you to execute all downloaded files from the directory they are downloaded to, immediately after downloading. As a geek, you surely can think of situations where this will come in handy. I have. :)
The sample above downloads allfiles from , executing them one-by-one as they are downloaded. The Sync configuration below downloads all and files from the same webpage, then extracts the zip files and executes the exe files. Note that only the exe files downloaded directly by Syncaid will be executed. It will not extract an executable from a zip file and then execute that executable.
The overwrite option
The overwrite option enables you to enable or disable overwriting existing downloaded files of the same name. By default Syncaid will always download the version at the download URL, even if it is not a newer version. Hey, the point of Syncaid is to synchronize, which means it expects to download!
There are rarely situations when the newer version of an installable application will use the exact same filename. More often than not, new versions include a version number. However, in some situations, for example, with Filehippo, you may be using a synfig that attempts to download several files and only really want the newer stuff. Re-downloading pre-existing files, especially when they run 20mb or even 200mb each, can waste quite a bit of time and bandwidth.
The overwrite option attempts to resolve that issue. Set the overwrite option to 0 (zero) and it will skip downloading and replacing files that would be named the same thing.
The assume option
The assume option is used primarily with the overwrite option, to provide a means of assigning an extension to downloaded files to ensure they are properly matched with an existing file.
This option can have multiple values, separated by a pipe character (|) or comma (,). If the multiple value syntax is assumed, a file of the same name and any of the matching types will effect a skip if the overwrite option is disabled.
The filehippo option
Perhaps one of the most effective software distribution sites for keeping application versions and download sources up-to-date is FileHippo.com. The beauty of FileHippo's sharing through Syncaid, is that you need only know the key to an application to enable it as a Sync. Enter the keys (multiple can be separated by commas), using the format below:
The filehippocats option
The filehippo option above demonstrates how to sync a single package using FileHippo.com as the versioning source. The filehippocats option is very similar, with the difference being that instead of selecting a single application or package to download you are selecting an entire category. Syncaid will check SaferPC.info for the current listing, and then enqueue downloads for each of the associated packages. Enter the category codes (multiple can be separated by commas), using the format below:
The sourceforge option
SourceForge.net is the most popular open-source collaboration website. Among the far more than 200,000 open source projects on SourceForge are some of the most popular applications, such as phpMyAdmin, 7-Zip, FileZilla, VirtualDub, Notepad++ and others. Syncaid takes advantage of the well-defined structures on SourceForge to enable Sync of one or more packages, based on the project name and, optionally, a regular expression pattern of the packages you wish to synchronize. Note that by default the package-matching pattern will match only exe and msi packages. Enter the keys (multiple can be separated by commas), using the format below:
Pattern-matching for release packages is accomplished by using a colon to separate the key from the pattern. The most recent packages matching the pattern will be Sync'ed. The following pattern downloads only release versions (not beta) of DQSD:
This one downloads release versions (not beta) of DQSD as well as the English-only phpMyAdmin "zip" package.
The microsoft option
Perhaps one of the most difficult sites to navigate and locate content on is Microsoft.com. Instead of a friendly name, download packages at the Microsoft Download Center are tagged with a GUID, a 36-character alpha-numeric character that has all but no meaning. Remembering which package a GUID is assigned to is near-impossible, and Synchronizing the most current versions of MS Download packages for offline installation has been an act of insanity. While Syncaid won't make it all better, it will enable you to setup a Synfig specifically to those packages you wish to monitor and Sync only updated versions. You'll have to find out the GUID in advance from the Microsoft Download Center, but then you can use the GUID in your Synfig to update any packages that do not require Windows Validation. Enter the keys (multiple can be separated by commas), using the format below:
The includes option(s)
Up to this point, I've treated you to the simplistic single-configuration-within-a-Sync-configuration-file approach. includes are the means by which you can chain multiple Sync configurations and/or multiple Sync configurations files together. Don't worry, it keeps track of what configurations were previously loaded, so it's safe to "loop" them without bringing the entire Internet to a screeching halt. ;)
Includes are processed within each configuration file whenever a configuration file is loaded, unless the specific include or Sync section has already been processed. A single configuration file can include multiple Sync headings, as long as they're named differently. Sections named anything other than the name of the configuration file are not loaded unless they're called by an include directive. Here's an example:
This Sync file downloads thefiles from ; then downloads the and files from , extracting each of them.
Here's a series of four Sync files, the first named
You can activate any individual Sync file by dragging and dropping it onto Syncaid. You can also rename the Syncaid executable to another filename, such as 3stooges.exe in order to change the default associated configuration files.
With the above Sync files you could drag and droponto Syncaid - and nothing will happen. The reason is that there isn't a section within the file, and there isn't an section either.
However, if you were to dragonto Syncaid, it would download and extract the "poke-y" rar series from the poke section (which was instantiated by the ), then it would load and sections from , in succession - renaming the fingers.txt file to a batch file, and extracting the files from to a child directory named . By instantiating first, the file wasn't touched, nor was the file.
If we instantiatefirst, , and are each processed in succession, though still has no effect directly, until called by includes in the file.
includes are clearly very powerful, but with great power comes great responsibility. It's important that you include a numeric includes values within the includes block to indicate how many includes you want to include. Then, for each include you need to name the value include# where # is the incremental numeric value of the specific include. They should be in the form of, , and so on. Failure to properly label your includes will result in either nothing happening, or nothing happening. In either case, you won't see very much happening. ;)
On the Web
You do not need to do anything special in order to enable Syncaid to work with your site, but you can use some options on anchor tags in order to facilitate a more user-friendly experience.
The most important thing for your anchors is to have a useful file name. In most URLs the filename is the right-most portion of an address, following the right-most slash (/). For example, for the URLthe filename is . Syncaid will correctly detect this filename from a link to that resource, and name the file appropriately when saving it to your system.
However, some sites use click-tracking or other funky URLs to represent standard files. In these instances, a URL may either contain invalid or unfriendly characters, or simply not include a filename at all. For example,or . In some of these instances Syncaid will correctly parse the filename on its own, but it's safer to link to the resource using a descriptive attribute, which can be parsed for the purpose of obtaining the correct filename.
To do this, you need only include a rel attribute for your anchors, with a saveas property wrapping the filename. Here's an example anchor:
rel="saveas[spiffy.jpg]">My spiffy file</a>
Syncaid will see this URL and download the resource addressed at the href as.
To do this, you need only include a rel attribute for your anchors, with an extract property (the "1" is required). Here's an example anchor:
rel="extract">My compressed file</a>
Syncaid will see this URL and download the resource addressed at the href, then extract the compressed file to the same directory. Note that the extract functionality requires 7-zip to be installed and within the system variable.
To do this, you need only include a rel attribute for your anchors, with an exec property (the "1" is required). Here's an example anchor:
Syncaid will see this URL and download the resource addressed at the href, then execute (run) it.
How to contact me
If you like Syncaid, or want to see a new feature added, please drop me a line. You can reach me by email, for questions, concerns, feature requests and so on. I am a very busy man, though, so I cannot guarantee that a feature requested will be added, nor that I'll respond to your message. I will try, of course.
|Shawn K. Hall © 2003-2021||Powered by 12 Point Design|