Difference between revisions of "OpenSuse"
Line 306: | Line 306: | ||
These work with Python 3. | These work with Python 3. | ||
== Complete Data Visualization Chain == | |||
Where needed, to accompany Pandas we also need MongoDB. This is not a standard package, but it is available from the Opensuse repository. For manual installation | |||
zypper addrepo http://download.opensuse.org/repositories/server:database/openSUSE_Leap_42.2/server:database.repo | |||
zypper refresh | |||
zypper install mongodb | |||
Revision as of 20:53, 11 February 2017
The observatory's servers and control computers run on the OpenSuse distribution of Linux-based software. Most current systems are using release Leap 42.1or 42.2. Our evolving installation notes given below are rewritten as we gain experience with the most recent releases. Except for older Nvidia video cards, install an Nvidia driver after the default installation and remove or disable the Nouveau community driver. With that exception, the installation from a USB iso image will work.
The following describes how to build a system with OpenSuse that provides a solid foundation of software for physics and astronomy for real-time control of telescopes and observatories, operating small servers, and processing astronomical data.
Before Installation
Prepare a DVD or a USB memory stick with the ISO image of the 64-bit distribution. OpenSuse's imagewriter is a convenient way to create the correct structure on the USB device.
On a new system not using RAID, deselect RAID in BIOS if it is offered. This will prevent OpenSuse from creating disk partitions with RAID. However, if RAID information has already been written to the disk the OpenSuse installer will assume a RAID configuration even if hardware raid is not enabled. A simple cure is to install the system twice. On the first pass use the Expert Partitioner option and delete the proposed raid configuration. Then in /dev/sda (or equivalent) add a root and a home ext4 partition but intentionally do not add a boot partition. The installer will warn you this will not work. Ignore those warnings and let the installer prepare the disk. Once that is accomplished you can abort the installation, or let it run to the end. The disk will not be bootable but it will be cleaned of RAID and on the next installation pass you will have a proposal to use the full disk with conventional structure and btrfs for the root partition.
For most new machines allow UEFI (custom option, if available) and disable compatibility mode. The installer will identify the system as allowing UEFI and properly select the boot configuration. However, also use the BIOS setup to change the boot priority to the medium reflecting this choice. The boot medium and a UEFI installation must match.
Opensuse will detect and set up a UEFI boot protocol unless this option is turned off in the BIOS. With that selection it will handle and format large disks.
Some recent hardware, notably the Supermicro X10-SRA, may hang on booting with older USB devices attached. While we do not know the cause, the cure in this instance was to enable EHCI-Hand-off in the USB configuration options presented for the BIOS. This may apply only to specific applications, and could be kernel-dependent,. In general, the default BIOS settings are fine for installation and need modification later if specific applications raise issues.
During installation
Insert the medium, reboot the system, and select Installation from the splash screen. If there is a booting problem, use the keyboard to bring up a boot selection screen (often "Del", F11 or F12), and check the boot order and if needed also the BIOS setup.
At this point if the system has a recent Nvidia card it also may be best to disable modeset. The symptom this is necessary is that subsequent booting freezes before the installation begins. With Leap 42.1 edit the boot options by pressing "e" before the system tries to start an installation. This will open a simple boot editing screen with instructions.
At the end of the line for linux add "nouveau.nomodeset=0" . Similarly, a problem with an Intel graphics card that was switching, perhaps to a Displayport interface, was fixed with simply "nomodeset".
Continue with the installation as instructed on this editing screen. The default settings should work with the following additions and exceptions.
Deselect software by taking the checkmark off with a spacebar press. After installation is complete, return to the software menu of YAST and make sure that those items never to be install (pk-update is the worst of them) are marked "Taboo". For now, just do not install them.
Select a user interface of "Other" rather than KDE or Gnome, and then Xfce as default environment to have a lightweight but fully functional system.
Select almost all packages by group except Apparmor which should be marked "taboo". Include PHP, MySQL, and Apache unless not needed not needed for your use. Decline KDE and Gnome desktop but add their development code. Some KDE and Gnome applications may be loaded individually later. Do not install laptop tools unless you are configuring a laptop. Otherwise network management will default to be selectively controlled by the user rather than by the system at boot time.
Set the computer system clock to use UTC, check the time zone and the local time.
The gparted package may be useful to manage disks larger than 2 TB. As of Opensuse 13.2 with new disks the installer will use BTRFS for the root partition and XFS for the remainder to make full use of large disks on UEFI systems. In the event of a failure, leaving a critical disk formatted in the wrong size or filesystem, add gdisk from a repository and reformat the disk. Reboot, and re-install the operating system on the reformatted disk.
While it would be preferred to use 100 GB for the root directory in a BTRFS, Opensuse by default allocates a marginally large enough 40 GB. It is sufficient for the base system, and if large files are needed in /usr/local/ they can be located in the /home partition with a soft link from /usr/local. The remaining space on the system disk will be formatted as XFS. An advantage to putting locally installed software in /usr/local/ and having that outside of the root partition is that in subsequent upgrades there is less risk of losing special software installations.
Deselect and mark "taboo" Apparmor for systems which do not require its access controls.
Add nano so that you will have a simple terminal-based editor after booting the first time
Delete pk-update to avoid nagware about package updates and mark it for non-installation permanently by selecting "taboo"
Turn off firewall (assuming your system is already behind an institutional or local firewall)
Open port for SSH
Check the boot option for grub2 on a non-UEFI system
Complete the installation from the media (either USB or DVD)
Remove the medium, reset the boot priority to the hard disk, reboot
From OpenSuse using YAST
Start yast from the command line as su with yast --qt
Disable DVD or USB in software respositories
Unless doing GPU development or you have recent nvidia hardware, do not include the repository for nvidia (creates a long term maintenance problem) and use the Nouveau Xorg driver
Perform all updates based on default respositories as needed
Note that in removing packages, as of OpenSuse 13.2, select Options --> Cleanup when deleting packages to prevent their automatic reinstalling though the preselection feature of Yast. Generally it is not necessary to remove packages unless there is something about them that interferes with your use of the system. In most cases they may be disabled in subsequent system configuration.
Remove really annoying pk-update-icon if you missed deleting it initially. You will have to mark it in YAST for permanent deletion.
Add Nvidia respository ftp://download.nvidia.com/opensuse/13.2/ if needed and nvidia graphics/gpu drivers
Add lsb5
Add apache if used as web server
Add php and packages if used as web server
Add gsl and gsl-devel
Add nano
Add timidity
Add audacity
Add audio-recorder
Add celestia (kde-celestia)
Add stellarium
Add galculator
Add gedit
Add gnome-disk-utility (previously palimpsest)
Add gnumeric
Add gtkglext-devel
Add hdf5 (required by Python Pynpoint-exoplanet)
Add hdf5-devel
Add hdf5-devel-static
Add imagewriter
Add libatlas3 (optional required by astromatic software)
Add libatlas3-devel (optional required by astromatic software)
Add lua-devel
Add mlocate
Add nasm (used by openh264)
Add netpbm
Add libnetpbm-devel
Add okular
Add pavucontrol (pulse audio control to work around problems with defaults)
Add plplot
Add plplot-devel (optionally other plplot packages as needed)
Opensuse Leap installs python 2.7 and python 3.4. The default system python command in /etc/alternatives points to python 2.7, but the default "pip" points to python 3.4. The following pacakges will go to Python 3.4. Equivalent packages are available for Python 2 without the "3" in the package name. An end user running "python" needs to explicitly call "python3", or change the alternative link.
Add python3-Beautifulsoup4
Add python3-Cython
Add Python3-Sphinx
Add python3-cairo-devel
Add python3-certifi (optional, may cause other issues)
Add python3-dateutil
Add python3-distutils-extra
Add python3-idle
Add python3-matplotlib and related packages
Add python3-numpy
Add python3-numpy-devel
Add python3-qt4
Add python3-qt5
Add python3-scipy
Add python3-scipy-weave
Add python3-sympy
Add libevent-devel
Add fftw3-devel, fftw3-threads, and fftw3-threads-devel
Add xfig
Add ufraw
Add gimp-ufraw
Add gphoto but not gphotofs
Add qiv
Add luvcview
Add motif
Add motif-devel
Add motif-devel-32bit
Add other motif libraries if they are not installed by default
Remove all virtualbox rpm's installed from OpenSuse
Add yasm
Add yasm-devel
Add libpng12-devel
Add libpng16-devel
If using Grace earlier than 5.1.25 deselect libpng16-compat-devel and select libpng12-compat-devel
Add fxload (used by SBIG cameras)
After updates
Disable modemmanager because it interferes with serial ports used for instruments
Disable avahi as unnecessary in our environment
Edit /etc/sysconfig to set locate default search to root
Use YAST to set NTP servers for your domain rather than Opensuse's defaults
From source in /usr/local
For rpm packages use zypper --non-interactive install package.rpm or add --no-gpg-checks if necessary.
Install nedit from rpm to /usr/bin and link to /usr/local/bin
Add lame and lame library packages for mp3 audio
Install mplayer through the command line svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer or from a stable package along with skin and codecs
Install ds9
Install xpa
Install cfitsio and manually copy lib64 and include installation directories to /usr/local/lib64 and /usr/local/include
Install grace (build from source with local FFT modifications for normalization)
Install ImageJ (separately from AstroImageJ below if needed for other features)
Install Aladin
Install xephem
Install astrometry.net
Install swarp
Install sextractor
Install psfex (current release does not build in Opensuse Leap due to cblas package incompatibility)
Install moodle (depends on mysql, apache, and php) on educational servers
Install mediawiki on servers as needed
Python
For OpenSuse Leap both Python-2.7 and Python-3.4 are installed. By default /usr/bin/python points to Python-2.7, but pip and easy_install will have softlinks in /etc/alternatives pointing to Python-3. We are changing from 2.7 to 3.4 and in current installations this is left as is, and Python-3.4 packages are installed (see Yast). However, to use 2.7 instead, remove the softlinks in /etc/alternatives and redefine them to point to /usr/bin/pip2.7 and /usr/bin/easy_install-2.7 so that these commands will work in our environment. This may have to be redone whenever there is a software upgrade.
In the case of using 2.7, if missing cacert files are reported when running pip or easy_install, be root user to remove the certifi package with the command line "pip uninstall certifi". In Opensuse Leap 42.1 the certifi package modifies the default search paths for cert files, and does not provide the certifi versions in a location that setup.py will find them. Removing certifi fixes the issue. This may not be the case with 3.4, which does not install certifi by default.
If on installing new packages with pip there are messages that Cython is out of date
Upgrade Cython (pip install --upgrade Cython)
Install pyephem (pip install pyephem)
Install stscipython with astropy using pip install stsci.distutils then pip install stscipython or
Alternatively install astropy or
Alternatively install pyfits and pywcs using pip install pyfits and pip install pywcs
Install healpy (healpix)
Install reproject (image reprojection for fits conversion)
Install quantities (physical constants)
Install emcee (MCMC)
Install pyastronomy (pip install pyastronomy) or from source on github pyastronomy
Install scikit-image (pip install scikit-image).
Optionally install Pynpoint-exoplanet (pip install Pynpoint-exoplanet)
Configure matplotlibrc in /usr/lib64/python3.4/site-packages/matplotlib/mpl-data/matplotlibrc for the GTK3Cairo backend. Since this default configuration may change on upgrades, each user should also have a copy of their preferred configuration in their .config/matplotlib/matplotlilbrc . The GTK3Cairo backend is compatible with the Xfwm and IceWm window managers and with the Xfce desktop that is on most of our systems.
Lastly, install the software chain for data visualization with Python
Install pandas (pip install pandas)
Install scrapy (pip install scrapy)
Install requests (pip install requests)
These work with Python 3.
Complete Data Visualization Chain
Where needed, to accompany Pandas we also need MongoDB. This is not a standard package, but it is available from the Opensuse repository. For manual installation
zypper addrepo http://download.opensuse.org/repositories/server:database/openSUSE_Leap_42.2/server:database.repo zypper refresh zypper install mongodb
From local software in /usr/local
Install AstroImageJ
Install AstroCC
Install Alsvid
Install xmtel (if needed)
Install xmccd (if needed, also provides libcfitsio and xpa)
Install hp15c
Install jedit
Optionally install astrometry.net
Update /etc
Copy motd
Edit HOSTNAME
Add entries to /etc/rc.d/boot.local
Add profile.local
Edit /etc/dnsmasq.conf as needed
Settings for the network
Configure network as needed for additional cards defined for internal zone
Configure dnsmasq as needed to service one or more cards
Add masquerade to firewall settings if internal zone present (required for dnsmasq ip forwarding)
Start the firewall if using dnsmasq or needing the security it provides
Start dnsmasq
Run services manager and turn off unused services
Run lsof -i to confirm there are no insecure open ports
Reboot the system
With Opensuse 13.2's use of the wicked network daemon, a configured network device will not show its IP until it is physically connected to an active network. The yast configuration option "at boot time" for network configuration means that these ports must see a live connection when the system is booted to find their configuration. This is not a bug, it is a "feature". The alternative option "on cable connection" is not useful for a fixed instrument controller. If a device is physically connected and does not show its IP in ifconfig, try "systemctl restart network.service" or a reboot.
Desktop
Run nvidia-settings to set display for a system with Nvidia hardware if the Nvidia drivers are installed. The latest community Nvidia support is adequate for most purposes without installing the proprietary Nvidia driver and kernel module. The system is more easily maintained if it runs using the community supported package which is improving quickly.
OpenGL with Nvidia
Users should be members of the video group to have access to opengl applications. If they are not, the application may run slowly (glxgears) or crash (celestia). For some applications with older hardware the Nouveau open source driver will suffice and be less likely to interfere with system updates later. This driver is compatible with randr and allows command line setting of multiple displays. For example if there are two displays on the graphics card, a command line such as
- xrandr -q
will list the available displays and their capabilities, while one such as
- xrandr --output DVI-I-2 --right-of DVI-I-1
will configure them as one screen providing acceleration across the desktop.
Newer Nvidia cards and all of the Quadro family require loading the lastest nvidia driver and the kernel modification. Add Nvidia as a repository and use YAST to manage the updates. Reboot the system afterwards.
Flash and Freshplayer
Do not install the proprietary and outdated Adobe Flash that is offered in the distribution. Remove it if it has already been installed before doing a system update.
Flash is now blocked by default in Firefox and will be deprecated and blocked in Google Chrome by the end of 2016. For those websites still using Flash, install Google Chrome for web browsing. It supports remaining non-complaint web flash usages and defaults to HTML5 when possible. Chrome has its own version of the Flash library for this purpose that Google keeps up to date.
Install the Freshplayer plugin (available from the distribution) to provide Flash support for Firefox. When Firefox is started it will find the version of Flash installed by Chrome and provide the translation for Firefox.
gPhoto2
The gphoto2 application runs Nikon DSLR cameras for real-time observing, scripted imaging, and called by cgi routines from a web server. To give the USB device the proper permissions without invoking unwanted software (the default for a Gnome installation in OpenSuse), we make sure that libgphoto2 is installed, but not the file system. In OpenSuse 13.2 there will not be a udev rules file installed by default.
As root user,
cd /etc/udev/rules.d
/usr/lib64/libgphoto2/print-camera-list udev-rules version 175 group video mode 0666 > 90-gphoto.rules
where the version given has to be high enough to work with udev and still be recognized by libgphoto2.
Add the video group to users who will be observers, and to the user wwwrun by editing /etc/group or by using YAST.
When a camera is connected or turned on, it will accessible by any user in the video group, including the cgi applications used for remote operations.
exFAT
Add fuse-exfat from OpenSuse package search, currently version 1.1.0
- zypper --non-interactive install fuse-exfat-1.1.0-2.1.x86_64.rpm
This provides support where needed for SDXC memory cards through the Microsoft exfat filesystem.
VLC
The version of VLC that can be installed with Yast lacks all proprietary codecs necessary for many common uses. The OpenSuse version should not be installed. To build from source --
- Install lua and lua-devel if not already installed
- Download the latest source tarball from VLC (currently 2.2.1)
- Use the latest openh264 source tar file, compile, and install
- Use the latest ffmpeg source tar file, compile, and install
- Untar ffmpeg
- ./configure --enable-pic --libdir=/usr/local/lib64 --enable-libmp3lame --enable-libopenh264
- make
- make install
- ldconfig
- Untar vlc
- ./configure --disable-mad --disable-a52
- make
- make install
Mplayer
- Install the source code in /usr/local/src/ --
- svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer
- Untar the codecs and skin files into /usr/local . We use a collection saved in mplayer_codecs.tar.gz that installs into share/mplayer and lib/codecs
- In the source directory, ./configure --enable-gui then make, make install
VirtualBox
VirtualBox as supplied by OpenSuse cannot be updated using the Oracle site. Instead of installing their version, we use the latest Oracle RPM which is currently version 5.0.14. Version 5.0 and higher supports USB3 in the host OS, and is therefore advisable for camera or storage drive use.
- Set the BIOS to allow virtualization technology and to allow advanced I/O for sharing resources.
- Retrieve the packages from https://www.virtualbox.org/wiki/Linux_Downloads .
- zypper --non-interactive install -no-gpg-checks install VirtualBox-5.0-5.0.14_105127_openSUSE132-1.x86_64.rpm .
- Retrieve the extension pack from Oracle's download site.
- VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-5.0.14-105127.vbox-extpack.
- In Opensuse YAST, add the Virtualbox guest kernel modules and guest tools, or use the guest additions from Oracle.
- As root run /etc/vboxadd-service .
- Create a directory that will be shared with the guest OS and set this up in virtualbox when building a virtual machine .
- Once the guest OS is installed, add the guest additions to it also, to enable the shared directory and mouse/pointer integration .
- Add the virtualbox group to the user(s) who will run it .
- Start the qt interface from the command line with virtualbox .
- Lastly, read the Virtualbox on-line manual .
For access to the USB system the guest OS must have a driver installed. Virtualbox presents a virtual xHCI USB3 device to the guest. The driver provided by Intel has worked for us in a Windows 7 installation.
OpenGL
Users must belong to the video group to have access to OpenGL when NVidia drivers are in use.
Skype
Skype is supported on Linux through its new Alpha version under rapid development. It integrates with the web version of Skype, and should soon allow the same features as the older obsolete Skype. Alternatives which are very good and also cross platform for video conferencing are Ezuce and Zoom. All of them offer a selection of audio devices within the application, but pavucontrol is also useful when the applications fail to detect the preferred hardware.
Wireless
Laptops by default will have networkmanager running their hardware and wireless connections. Desktops will not. To enable desktop wireless with minimal need for configuration, use Yast, Network Settings, and Global Settings to select networkmanager rather than wickedd. With that change, there will be a desktop icon in the system tray and the interface may be selected by the user.
Few USB network adapters work with the Linux kernel in OpenSuse 13.1 . Only one we have found readily available new is the Buffalo Nfinity Wireless-N compact USB 2.0 adapter. It is recognized immediately and requires no additional configuration, other than the selection of networkmanager, and the user's choice of connection.
Static LAN and dnsmasq
We use dnsmasq to manage local area networks (LAN) from a second network device on telescope computers. Typically the device address is set to 192.168.0.1/24, or to 1.1/24 if there is another LAN operating. The configuration file for dnsmasq is set to point to the device, i.e. eth1, to which the switch is attached.
This works well if (a) there is a switch attached and turned on, and (b) the computer is running the wickedd manager which is the default in current Opensuse releases based on systemd. It is seeming not possible, or certainly not straightforward, to run a lan from a laptop which is configured with networkmanager.
To attach a networked instrument such as a camera to a laptop that by default is configured with network manager the options are
- Attach the device to a switch which itself is integrated into a LAN with DHCP provided by another computer system
- Change the laptop networking to run wickedd instead of networkmanager.
The disadvantage to the second option in the laptop world is that wickedd does not have the end-user support for wireless networking that networkmanager provides. Further, when switching from one system to another, there are inevitable configuration issues, particularly with the management of host resolution and the file /etc/resolv.conf.
The basic process is to use yast or yast2, select network device configuration, and change the manager to wickedd. This will allow editing the individual network devices. Set the static ip address for the device that will handle the LAN, edit the device entry, change it to "internal", and set it to activate on boot through the setting in the Global tab. Shutdown and reboot the system. The ethernet adapter must be inserted at boot time.
As superuser use "wicked show all" to see the status of the devices, or "wicked ifstatus eth1" to see the status of one network device. Each device has a configuration file in /etc/sysconfig/network/, such as ifcfg-eth1 for eth1. Within that file there should be a line which says
LINK_REQUIRED=no
As of Opensuse 42.1, this line is not inserted by the yast2 configurator, and consequently the network device will stall and wickedd will report "setup-in-progress". The simple solution is to enter this by hand if you see this error and need a second network active on power up.