Support - Center for Cybersecurity and Cyber Operations
We value your feedback, questions and trouble reports. Please contact me at: mfthomps at nps.edu. In addition to the items noted below, our GitHub "issues" describes some known problems, and is a good way to report discrepancies. A brief demonstration of Labtainers installation and use is here.
If you'd like to be notified of updates to the Labtainer framework, please send an email to labtainers-request [at] freelists.org with 'subscribe' in the Subject field OR by visiting our mail list page.
Troubleshooting suggestions are listed below. You can update your Labtainers installation by running:
from your labtainers-student directory.
Labtainers runs on x86-based computers (not ARM) running Windows, Mac or Linux. The new Apple M1-based laptops will not run Labtainers. The Labtainers VM appliance will consume about 30GB of disk.
The most common problems are due to your VM losing its network connection. If you cannot use the VM's browser to access the internet (or if wget google.com fails), your VM network is failed. Either reboot the VM, or use "
sudo systemctl restart network-manager" to restart the network.
Virtual machine troubleshooting (problems with the VM appliance)
- The "student" password on the VM is "password123".
- VMWare virtual machine windows sometimes become hard to use, depending on your platform. Try using: "View" / "Autosize" / select "Autofit Guest".
- If you have trouble installing a 64-bit guest on VirtualBox on Windows 10, refer to this link:
https://forums.virtualbox.org/viewtopic.php?f=1&t=62339 Or, run
bcdedit /set hypervisorlaunchtype offfrom an elevated command prompt.
- If VMs will not boot, confirm you have enabled VT-x/AMD-v in your BIOS https://forums.virtualbox.org/viewtopic.php?f=1&t=62339
- If VirtualBox fails to import the appliance, they give the same error code regardless of the reason (thanks Oracle!). Make sure you have at least 30GB of disk on the drive hosting the "base folder".
- If a VMWare virtual machine becomes unable to reach your network, this can usually be resolved by rebooting your system, e.g., your Mac. Alternately, these commands often work from within a terminal on the Mac:
- sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --configure
- sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --stop
- sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli –start
- When first starting a VM on Mac OSX, if the VM seems to freeze, i.e., you cannot type or click in the desktop: use preferences/security/unlock/privacy "Allow apps below to control your computer"
- Ubuntu Package manager error (red stop symbol in upper right of desktop): If an Ubuntu VM reports package errors, try using the "sudo apt-get install -f" command and then reboot.
- If your VM disk fills up, e.g., the "df" command shows Use of 100% on the root file system, you can reclaim space by deleting the current lab (or some other) from the labtainer-student directory:
removelab.py <labname>(this will cause you to lose work)
- If you "opened" the download link rather than downloading the file, your computer may have changed the name of the file from .ova to .ovf. You will need to change the file name back to ova before you can import the appliance.
- If a VirtualBox VM network becomes outrageously slow, power off the VM and change the network adaptor type in Settings / Networks / Advanced. Yeah, that sounds stupid, but it often works.
- If a GUIs fail to open (e.g., firefox or wireshark), and you see an error such as "cannot open display: :0", your X11 socket on the VM may have been corrupted. On the VM, an ls -l /tmp/.X11-unix should display a socket named "X0". If it does not, you may need to reboot your VM.
- Ubuntu 22 and other recent Linux distributions having systemd 248 and later will not work with Labtainers.
- Get updates by running update-labtainer.sh (These include fixes to student and instructor environments.)
- GUI windows may sometimes display as solid black boxes. Resizing the window, or closing it (e.g., via Esc) and then reopening it will usually force the window to repaint.
- If labs are unable to reach the Internet, e.g., to ping google, and you are connected to a VPN, try disconnecting the VPN.
- The CyberCIEGE lab on VirtualBox running on OSX Catalina is not usable due to mouse problems as described here. https://github.com/mfthomps/Labtainers/issues/39
- When starting a new Labtainer, if you see an error such as "No JSON object could be decoded", try updating your installation (per above).
- If you see this error: "Remote image ... is lacking a base version, it needs to be retagged with trunk/distrib/retag_all.py" run update-labtainer.sh (from the labtainer-student directory) to update your distribution.
- If updates fail or nothing seems to work, your installation may have been corrupted. You can recover from that as follows:
If a Labtainer computer is unable to reach the Internet, though the VM is able to -- and the Labtainer computer is configured to use a Docker gateway (per the routing table), then your Linux kernel may have bit rot. Try doing an "apt-get upgrade" on the VM and then reboot.
- Docker Desktop is not currently supported due to its instability. Suggestions below are provided for those who wish to experiment.
- OS X (Mac) may give a permission or "mounts" error. Update your Docker Desktop version.
- OS X Now use a GRPC Fuse that seems to prevent sockets (e.g., for X11) on mounted directories from propagating. Use the Preferences in Docker Desktop to revert back the osxfs.
- Cannot access external networks from Labtainer containers.
- X11 applications may fail due to failures in sharing sockets with containers created by the master container. There seems to be no problem sharing sockets between the vnc container and the master, but when the master creates a container and names its volume, the socket is not functional.
- Get updates by running labtainer/setup_scripts/update-designer.sh
- Periodically use the /labtainer/setup_scripts/cleanupDocker.sh to remove unused images. This will reclaim disk space consumed when doing many iterations of building the same lab.
- As a last resort, use the labtainer/setup_scripts/destroy-docker.sh to fully erase all local Docker images. Then use pull-all.sh to pull in all baseline images (several GB download).
- If your built labs do not seem to initialize or run the way you wish, look at the /tmp directory on the containers for diagnostics.
- If your /tmp/parameterize.sh.log contains lines like "sudo: unable to stat /etc/sudoers: Permission denied" you likely copied a lab between systems, e.g., using a VM shared directory. That messes up file permissions. The best solution is to go back and copy lab files using tar and/or scp to retain permissions.
- If a Labtainer computer is unable to reach the Internet (e..g, during a rebuild), though the VM is able to, then your Linux kernel iptables may have bit rot. Try doing an "apt-get upgrade" on the VM and then reboot. Also check to see if multipass is installed, that may subvert iptables, preventing NAT on packets from a container.
Acknowledgment: This work was supported by NSF grant DUE-1438893. The views expressed in this material are those of the authors and do not reflect the official policy or position of the National Science Foundation, the Naval Postgraduate School, the Department of Defense, or the U.S. Government.