John Simpson <jms1@jms1.net>
2017-01-09
Part of my job involves managing a pair of VPNs, one for remote software developers and one for remote testers. One of the problems I wrestle with is keeping the software on the routers up to date, but only after I've had some time to test each new version. On top of this, a few of the users (C-level executive types who fancy themselves nework engineers) have SSH access to the routers in their homes, and insist on updating the software themselves - and of course they do it whenever they feel like it, whether I've had time to test the new software or not.
There are several "automatic update" scripts out there already. All of the ones I've seen rely on the "/system package update
" mechanism to actually update the software, but there doesn't seem to be a whole lot of information out there about how that mechanism actually works, particularly how to control what version it installs.
So my first thought was to check MikroTik's documentation. The only thing it had was a simple list of the commands under that menu, but nothing about the mechanism it uses "under the covers" to find out what versions are available, or how it actually downloads or installs them.
Next I tried searching the web for information about how it works. The best thing I was able to find was a post on the MikroTik forum which vaguely mentioned setting up an FTP server with the package files, and adding a static DNS entry within the router to make it use that server instead of MikroTik's server. Again, not enough detail for me to actually do anything with.
My next step was to ask on the Freenode #mikrotik
IRC channel, to see if anybody there knew how to set up. I've had pretty good results asking questions there in the past, but for some reason they people on the channel that day seemed more interested in chastising a "noob" for daring to ask questions without "doing their homework first" than they were in being helpful. What little information they did share ended up being either deliberately lacking, or just outright wrong. (I'm not naming any names, but if you've spent any time on the channel, you probably have a good idea who it was already - apparently they are well known on the channel for going out of their way to be a jerk, and in fact they like to brag about doing it.)
So over the weekend, when I had time to set up a test network at home and do some packet sniffing, I was able to figure it all out. I now have a private web server in the office, with a repository containing the software versions I have tested and approved. I've configured my remote routers to use this server, so when a user (or a future auto-upgrade script) runs "/system package update install
", they get the most recent version that I have tested and approved, rather than the most recent version MikroTik has released.
The /system package update
menu has a few "basic" commands, along with some "convenience" commands which combine one or more of the basics. The basic commands are:
check-for-updates
- Checks MikroTik's servers (i.e. a load-balanced Amazon S3 instance) to find out what the latest software version is.
print
- Shows the "upgrade status", i.e. what version is currently running, what new version is available, and the status message from the most recent check-for-updates
operation.
download
- Runs "check-for-updates
" if necessary. If the latest available version is newer than the currently running version, downloads the new version's ".npk
" files for the packages currently installed on the router. (If the router is using a routeros
combined package, this will download the new version's combined package.)
install
- Checks for updated software. If a newer version is available, downloads the relevant packages, reboots to install the them, and deletes the downloaded packages when it's finished.
The check-for-updates
operation, whether executed directly, or as part of another command, does the following:
Download http://upgrade.mikrotik.com/routeros/LATEST.6
, which is a one-line text file containing the version number and timestamp of the latest released version of RouterOS. For example...
$ curl -s http://upgrade.mikrotik.com/routeros/LATEST.6
6.38 1483097636
If the version is newer than the currently running version, download http://upgrade.mikrotik.com/routeros/6.38/CHANGELOG
(where 6.38
is the version number contained within the LATEST.6
file.) If this file doesn't exist or the download fails for some other reason, the check-for-updates
operation fails.
The results are held in memory for a certain length of time, but they appear to eventually "forgotten" by the router. (I'm not sure yet how long the results are retained, if I happen to find out I'll update this document.)
The download
operation, whether executed directly or as part of an install
command, does the following:
If the check-for-updates
status is not present, run that operation first.
Assuming a newer version is available, download the new version's .npk
files from the same http://upgrade.mikrotik.com/routers/6.x.x/
directory from which the CHANGELOG
file was downloaded, into the router's internal flash.
For example, if the router currently has the "system
", "ipv6
", and "security
" packages installed, the new versions of those three packages will be downloaded.
Note that if the router is currently running a "routeros
" all-in-one package, the new version's "routeros
" package will be downloaded.
This command fails if the router's internal flash storage doesn't have enough room to hold the downloaded files. However, any partially-downloaded files will still remain in flash, so if it fails because you don't have enough space, you should probably delete these files before trying again.
The install
command does the following:
If the check-for-updates
status is not present in memory, run that operation first.
If the new version's .npk
files have not been downloaded, download them.
Reboot the router in order to install the new software.
Notes:
The download
command does not automatically reboot the router. After downloading the files, you will need to reboot the router (using "/system reboot
", or "/system shutdown
" followed by power-cycling the router) in order to install the new software.
If you use the download
command to download software updates and then reboot to install the new packages, the downloaded files do not appear to automatically be deleted as part of the upgrade process. If you use the install
command, the files will be deleted.
The specifics of how to set up a web site will depend on what software you're running. Below is a very quick example using Apache, but this is NOT a complete example that you'll be able to copy and paste without understanding what you're doing.
Depending on which OS distribution you're using, there will probably be a default configuration of some kind, which in many cases will use /var/www/html
as the document root for a default web site. If the web server will only be hosting the one site containing these files, you can use that directory as your document root. Otherwise, you'll need to set up a virtual host of some kind, with its own document root.
Assuming you're setting up a virtual host, you will probably need a block like this, somewhere in your Apache configuration:
<VirtualHost 192.168.88.6:80>
ServerName upgrade.mikrotik.com
DocumentRoot /var/www/upgrade.mikrotik.com
</VirtualHost>
In this case, the site's "document root" is "/var/www/upgrade.mikrotik.com
". On the server, make sure that the "document root" directory exists and is world-readable (i.e. "chmod 0755
"), and then restart Apache.
At this point I suggest creating a tiny index.html
file in the document root directory, and using a browser to make sure that the site is working. Once you have verified that it's working, make the directory owned by a non-root user (i.e. "chown jms1:jms1
") and world-readable (i.e. "chmod 0755
"), and then log out of root. (You may want to remve the index.html
file, or replace it with something more useful for your users.)
routeros
directoryWithin the document root, create a directory with the name routeros
, with world-readable permissions (i.e. "chmod 0755
"). Everything involving RouterOS will be within this directory.
To add a RouterOS version to the site:
Make sure you know which architectures (i.e. "mipsbe
", "smips
", "x86
", etc.) you need on your server, in order to support your routers.
Within the document root's routeros
directory, create a directory whose name is the version you're adding (i.e. "6.37
", "6.37.1
", etc.) This directory should be world-readable (i.e. "chmod 0755
").
Build the URL for the version's CHANGELOG
file, and download it into the version directory.
$ cd /var/www/html/routeros/6.37.1
$ curl -O http://upgrade.mikrotik.com/routeros/6.37.1/CHANGELOG
$ chmod 0644 CHANGELOG
Download the "routeros-ARCH-VERSION.npk
" file for each architecture from MikroTik's download archive.
Upload the files to the DOCROOT/routeros/VERSION/
directory.
Download the "all_packages-ARCH-VERSION.zip
" file for each architecture from MikroTik's download archive.
Unzip the file.
Upload all of the "*.npk
" files from the zip file, into the DOCROOT/routeros/VERSION/
directory.
Create a LATEST
file in the directory, containing the version number and timestamp value for the release.
$ dd if=system-6.37.1-mipsbe.npk bs=1 skip=34 count=4 2>/dev/null | od -l
0000000 1475231321
0000004
$ echo '6.37.1 1475231321' > LATEST
Note: All of the system-VERSION-ARCH.npk
files for a given version should have the same timestamp value. If you find this is not the case, you should make sure you have downloaded, unzipped, and/or uploaded the files correctly.
At this point, the directory should contain a CHANGELOG
file, a LATEST
file, and all of the .npk
files for all architectures you need to support.
Fix the permissions on the directory and files.
cd /var/www/html/routeros/6.37.1
chmod -R go=u-w .
The last step is to create the LATEST.6
file in the routeros
directory. We created a separate LATEST
file within each version's directory so that switching the site from one version to another could be done by creating a symbolic link.
For example, to make the site tell your routers that the latest version is 6.37.1
, you would run a command like this:
$ cd /var/www/html/routeros
$ ln -sf 6.37.1/LATEST LATEST.6
The last step, of course, is to configure the router to use your server. The "upgrade.mikrotik.com
" hostname is hard-coded into RouterOS, but because the router uses itself as a nameserver, you can add a static DNS entry which makes this name point to any IP you like.
Assuming your private web server's IPv4 address is 192.168.88.6
, the command is:
/ip dns static add \
name="upgrade.mikrotik.com" \
address=192.168.88.6
Once this is done, any /system package update
commands will talk to your server, rather than talking to MikroTik's Amazon servers.