downloadplates.pl Instructions v4.4 Author This perl script was written by Mike Stewart mstewartga @ yahoo.com. Copyrite 2009 Mike Stewart Purpose This program was written to support the GRT EFIS plates functionality but can be used by anyone wanting local NACO approach plates and airport diagrams on their PC. Function This program uses the wget function to retrieve the NACO plates from the FAA website, and get them onto your computer so you can use em with your GRT EFIS system, or so you have have those plates locally for your use. License This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Setup Requiremnts 1. You must have perl installed. I recommend you run the active perl msi windows package installer 5.8.x or higher. Default settings will be fine. You can try to download it from here. http://www.activestate.com/activeperl/ . After you install active perl, you will need to install the xml package. After the perl install and reboot to get the environment variables set, At the command prompt type "ppm install xml-xpath" without the quotes. This will install the required perl xml parser. You need only do this once. 2.I have included a windows version of wget.exe with this package. This wget.exe is required for the script to work. You can just leave it in the directory where you run the script, or you can move it somewhere and add it to your path. I recommend you just leave it in the directory where you run the downloadplates.pl script. If you wish to move it and path it, generally this is done: From the desktop, right-click My Computer and click properties. In the System Properties window, click on the Advanced tab. In the Advanced section, click the Environment Variables button. Finally, in the Environment Variables window, highlight the path variable in the Systems Variable section and click edit. Add or modify the path lines with the paths you wish the computer to access. Each different directory is separated with a semicolon. Your path should look something like this C:\Perl\site\bin;C:\Perl\bin;%SystemRoot%\system32; %SystemRoot%;%SystemRoot%\System32\Wbem;C:\Program Files\wget ATTENTION non-windows users. You can delete the wget.exe from the package and use your own which you probably have already on your OS. Instructions Note. If you were running the getallplates.pl script before this..... DELETE everything in that directory including your plates. Delete everything from your USB stick you use for the EFIS TOO. This script changes the functionality completely and the old stuff will just take up space. This new script is highly efficient. PLEASE See the Usage section below BEFORE you start typing commands. 1. After you have the requirements above completed. You can execute the downloadplates.pl file from anywhere on your system. SEE USAGE BELOW. Just put the downloadplates.pl into a directory where you want to keep your plates along with the wget.exe file. I recommend you create a directory at your root called downloadplates. Then put these files in it. You can run the downloadplates.pl from there and you will end up with your plates in something like c:\downloadplates\plates NOTE: The first time you run this you will want to use the command "downloadplates.pl --forceupdate yes" w/o the quotes. This switch will get ALL the available plates for all the states the first time out. Assumming of course you actually want ALL the states which I normally do. 2.Open a command prompt. Start, run, cmd or something like that. 3.Navigate to where ever you have your put your downloadplates.pl file. Commands like "cd \" "then "cd downloadplates" or whatever. Just go to where you put the file downloadplates.pl 4. From the command line type "downloadplates.pl --forceupdate yes" w/o the quotes the first time you use this script to get all the plates. Or type "downloadplates.pl" which will get only the changed/added plates for this cycle and update what you have. 5. If you have not run "downloadplates.pl" for a cycle(28 days), meaning you have skipped a cycle, then you should run the "downloadplates.pl --forceupdate yes" to get current. See the logfile.txt for information on your last run of the script. 6. You can set up a task scheduled every 28 days if you like that will run this script automatically for you. Windows has a simple task scheduler that allows you to run any file at an interval of your choosing. Just tell the task scheduler to run downloadplates.pl every 4 weeks at night and your always up to date. How it works 1. The perl script will run and first go to the FAA NACO site and determine the current cycle number via html scraping. 2. It will then grab the latest cycle catalog I have available from my website. This has all the information necessary for every airport in the US to dounload the plates. This MAY NOT be the latest cycle!!!! I do my best to keep it at least within a few months up to date. 3. A plates directory will be created locally 4. Inside the plates directory, each airport identifier will have its own directory 5. Minimums plates are not downloaded by default. Approach plates, arrival and departure procedures, and airport diagrams are downloaded if they are available. 6. If not running the --forceupdate yes switch mentioned above, the script will look in the catalog for changes/deletions/additions since the last cycle and execute what is required to get you current for this cycle you in. 7. ~4gb of files are downloaded when getting all the states. Much much less for a cycle update only. 8. Getting all states can take upwards of 8+ hours on a typical broadband home connection the first time around. It can easily run in the background while you do other things. It is not resource intensive. But keep the computer on till its done or your will have to start over. I typically run it overnight and wake up to it being done. 9. Actual NACO pfd files names are not very useful. This script renames them into something that is useful, relevant and easier to read in the directories. This was done to support the functionality of the GRT EFIS and to be able to know what the darn thing is. 10. Once its done, you can copy the plates directory onto a USB drive for use with the GRT EFIS. To do this you can use the xcopy command from a command prompt. An example might be: xcopy c:\downloadplates\plates\*.* e:\plates /d /e /c /y /f The first time this will take a while since the stick is empty. It will prompt you to create the plates directory on the stick if you dont have one. The next time you do this it will only copy whats changed. USB is a bit slow so this xcopy command makes it nice for updates to be speedy. 11. A logfile.txt is appended each time the script is run to provide useful historic information. You can open it and see your history of usage. Usage As of version 2.1, I have added some input parameters to the command line. Running the simple command downloadplates.pl will produce the following default activity: 1. All states will be used and I assume you only want to update what you have with changes since the last cycle. 2. If a plate has been changed or added, it will be downloaded. If marked by the NACO catalog as no change, no action is taken. Unless... 3. If it is not marked as add, changed or deleted, AND you dont have it at all, I assume you want it and you will get it. 4. If the NACO catalog has marked a plate for deletion, it will delete yours if you have it. 5. A directory from within where you run the script, will be created called plates. inside the plates directory will be directories named after the identifiers. Switches The following list of paramaters may be also put with the command line, in any order, to produce the desired behavior. These are calles switches --debug yes Will put the script in debug mode. It will d/l the xml catalog, and parse it normally, but WILL NOT d/l the actual plates. It does extensive logging in your logfile.txt that can be used to debug issues. default is no. --states xx,xx,xx You can list the states you want. No spaces in the state list. Default is all states --Volumes xx,xx,xx There are a list of volumes at http://naco.faa.gov/content/naco/images/TPP_Index.gif . If you want to d/l a volume set of states, input the volume name. Default is all if there is no --states parameter is used. No spaces when seperating volume names. --destination drive:\dir This allows the user to determine where they want the plates directory to be located. This could be your usb stick for example. Default is to put the plates directory in the directory where you ran the script from. --forceupdate yes Will d/l the plate regardless of whether or not the NACO catalog has it marked as an add or change. NOTE: If you have skipped a cycle, YOU SHOULD use this switch. Default is for it to d/l plate only if it has changed or added. But, if you keep up to date with the cycles, then do not use this option which makes the updaing process really really fast. Also the wget command that this tool uses does a pretty good job. If what you have is the same as what you are trying to get, it quits trying and moves on. default is NO and it will only d/l changes since last cycle. --help Will give you some very detailed help displayed:) --getmins yes This will download the minimums charts also. Default is to not get them. They are not much use I found and take up space --longfilenames yes The default filename is something like RNAV-GPS-RWY-10.pdf Using this --longfilenames yes parameter the new filename would be pre-pended with the state and faa airport identifier. The new filename would be RI-BID-RNAV-GPS-RWY-10.pdf --statefolders yes New in v2.8 the will write the plates into folders organized by state and then by city. If your getings these plates for the GRT system. Do not use this. For everyone else, I would use this switch as it will make your directory structure easier to navigate. Some examples: downloadplates.pl --states ri A nice way to start off to see what the script does. This will d/l only the state of Rhode Island. Runs fast and is a good way to see whats going on. downloadplates.pl --forceupdate yes Great for running the first time when you want all states. Will get all states, all plates, except the minimum plates. downloadplates.pl --states sc,ri --volumes se-1 --destination c:\myjunk\downloadplates will d/l plates from the se-1 volume set which is Tennessee and Kentucky, and the states Rhode Island and South Caroline, if in this cycle it has been tagged by the FAA NACO group as having changed or been added, and will put the airport identifiers into c:\myjunk\downloadplates\plates downloadplates.pl --states sc,ri --volumes se-1 --destination c:\myjunk\downloadplates --debug yes Will do everything in the previous example but it WONT actually d/l the plates and it will do extensive logging for you into your logfile.txt. downloadplates.pl --states sc,ri --volumes se-1 --destination c:\myjunk\downloadplates --forceupdate yes Will do everything in the 1st example except it will ignore what the FAA NACO group has identified as added or changed, and it will d/l the plate regardless downloadplates.pl --volumes se-1,se-2 --forceupdate yes will d/l plates from the 2 volume regions se-1 and se-2 into the plates directory where you executed the script, and get the plates regardless of whether they been added or changed. downloadplates.pl --destination e: --getmins yes --statefolders yes will d/l all states with plates that have been changed or added since the last cycle to your e:\plates, subdirectories by state, & you will get the minimums plates as well Issues: Alaska has an airport identifier NUL. Unfortunately windows and other platforms do not allow you to create a directory called NUL. This is a protected word. Sorry but right now you cant save plates for this airport. Future development A gui front end can be developed to allow for user selectable download and generally make it nice to use. For now this is a raw, get all the plates, or get changed. Other feature requests can be sent to me and ill prioritize them with other things in my life! There will probably be a 3.0 version. This 2.x gets the job done. You can visit http://www.mstewart.net/super8/grtgetallplates/ for information about this script. Acknowledgements Thanks to nacomatic for providing the jump start on FAA NACO plate functionality. You have laid the foundation for our use. Thanks to the team that developed wget. http://www.gnu.org/software/wget/ Thanks to my buddie Robert Mullins who debugged my regex's when I was in the weeds. History revisions 1.0 First release using the NACOMATIC batch files 2.0 Stopped using the NACOMATIC batch files in favor of the FAA's xml catalog file. 2.1 Added some command line input parameters --debug yes --states xx,xx,xx --Volumes xx,xx,xx --destination drive:\dir --forceupdate yes --help --getmins yes --startover yes 2.2 Added another input parameter --linux added a couple of folders \Time and \XML for the linux crowd to the standard distribution. These are not used on Windows platforms. 2.3 Fixed a critical defect in the wget command. The -c continue argument was being used. This should not have been used. Shame on me for not catching that sooner. Added another input parameter --longfilenames yes Removed input parameter --linux Script now determins the OS on its own Cleaned up the script a bit. Added some additions logging to the debugger 2.4 Added wget.exe to the package. Removed need for adding to the path. Added IP address function to speed things up a bit. Added file size counts 2.5 Added support for MAC's. Thanks John Ewing. 2.6 Corrected bug in wget variable assignment. 2.7 FAA changed the location of the URL for the files and listing. 2.8 Added option switch to put the plates into directories by state and city. While this would NOT work for GRT users wanting PDF, for everyone else that wants their plates on the system of choice, its probably a much better solution to have them organized by state then city. Removed Startover function. Too hard to deal with multi OS issues. If you need to startover just delete your plates directory and use the forceupdate switch 2.9 Cleaned up some old code. Fixed problem with slashes spaces and periods in city names for the statefolders options so directory writes dont get rejected. 3.0 FAA moved the xml file to a new location. Example http://www.aeronav.faa.gov/d-tpp/1106/xml_data/d-TPP_Metafile.xml 4.0 MAJOR change. The catalog file is no longer readily available from the faa. This is a big problem!! SO now I have to find it somewhere, store it on my server so you can get it. This is a sub optimal solution but it will work for now. 4.1 script now getting xml catalog from the FAA. Back to working as designed. 4.2 Had to change the file and directory locations for the FAA websites. 4.3 never released 4.4 More cat and mouse with the FAA XML catalog file. Still having trouble finidng it. However an almost current version has been found here. https://nfdc.faa.gov/webContent/dtpp/current.xml And they moved the pdf again. here is an example path of the new location http://aeronav.faa.gov/d-tpp/1503/01244R23.PDF