Using the Python Tools#
Several python scripts in this area can be used to emulate parts of the Percival data acquisition system. This is a summary of the tools and instructions on how to operate them. There are two options for using the tools:
Create a virtual python environment, then install packages and the tools into it.
Install required python packages and set PYTHONPATH to point at the appropriate location
The virtual environment is the preferred solution since it does not require system privileges to install packages.
NOTE The h5py python bindings require the HDF5 libraries to be installed on your system (see above). If
not installed into a standard location, set the environment variable HDF5_DIR to point to the installation,
prior to either installation method. See the h5py install instructions
for more information.
Installing a Virtual Python Environment#
Ensure you have virtualenv installed in your python environment first. Then create and activate a new virtualenv - and finally install the required dependencies with the following commands:
# Ensure you are in the "build" directory created above otherwise move into it
cd <project root>/build
# First create the virtual environment.
# This need to be done only once for your working copy
virtualenv -p /path/to/favourite/python venv
# Activate the venv - this need to be done for each shell you work in
source venv/bin/activate
# Move into the python directory to install
cd lib/python
# Install the required dependencies.
# This step is only required once for each virtualenv
pip install -r requirements.txt
# Install the tools into the environment
# The develop argument allows the underlying tools to be updated from the source in the
# build step without requiring the setup script to be run-executed each time
python setup.py develop
# Move back to the "build" directory
cd ../..
Having set up the virtual environment and installed packages and tools, it can be subsequently
reused simply by activating it with the source venv/bin/activate command. The virtual env can
be exited with the command deactivate at any time.
The setup.py script creates command-line entry points for the tools in the virtual environment,
which can be invoked directly, as shown in the descriptions below.
Installing the required python packages manually#
This option is less preferred since system privileges are typically required to install packages and it is necessary to ensure that PYTHONPATH is set appropriately. The following steps are required:
# Install required python packages using pip. This requires system
# privileges typically, e.g. sudo
# If pip is not installed in your python environment, packages can also be
# downloaded and installed manually: seek local expert advice for this if
# necessary
pip install posix_ipc
pip install pysnmp
pip install numpy
pip install h5py
pip install pyzmq
# Set the PYTHONPATH to point at the location of the tools
export PYTHONPATH=<project root>/build/lib/python
The pyzmq python bindings for ZeroMQ do not require ZeroMQ to be installed, although this is requried to
build and run the compiled applications. There is, however, a potential conflict between pyzmq and ZeroMQ v3
libraries on RedHat-based systems, causing python tools to fail when importing the bindings. If this occurs,
delete the bindings with the command pip unintall pyzmq and reinstall with
pip install pyzmq --install-option='--zmq=bundled' to force installation of a local ZeroMQ library.
NOTE: unlike in the virtual environment, this installation method does not create command-line
entry points for the python tools. In this case, invoking them requires python to run with a module
entry instead. For instance port_counters ... becomes python -m port_counters ....
Available python tools#
The following describe the available tools, and their configurations.
frame_producer
This tool generates a simulated UDP packet stream, allowing the FrameReceiver to tested without mezzanine hardware.
(venv) $ frame_producer --help
usage: frame_producer [-h] [--destaddr [DESTADDR [DESTADDR ...]]]
[--frames FRAMES] [--interval INTERVAL] [--display]
FrameProducer - generate a simulated UDP frame data stream
optional arguments:
-h, --help show this help message and exit
--destaddr [DESTADDR [DESTADDR ...]]
list destination host(s) IP address:port (e.g.
0.0.0.1:8081)
--frames FRAMES, -n FRAMES
select number of frames to transmit
--interval INTERVAL, -t INTERVAL
select frame interval in seconds
--display, -d Enable diagnostic display of generated image
NOTE: multiple destination address are supported.
For example, to transmit three frames total to frame receivers on three separate hosts:
(venv) $ frame_producer --destaddr 10.1.0.1:8000 10.1.0.2:8000 10.1.0.3:8000 --frames=3 --interval=0.1
Starting Percival data transmission to: ['10.1.0.1', '10.1.0.2', '10.1.0.3']
frame: 0
Sent Image frame: 0 subframe: 0 packets: 424 bytes: 2112368 to 10.1.0.1:8000
Sent Image frame: 0 subframe: 1 packets: 424 bytes: 2112368 to 10.1.0.1:8000
Sent Reset frame: 0 subframe: 0 packets: 424 bytes: 2112368 to 10.1.0.1:8001
Sent Reset frame: 0 subframe: 1 packets: 424 bytes: 2112368 to 10.1.0.1:8001
frame: 1
Sent Image frame: 1 subframe: 0 packets: 424 bytes: 2112368 to 10.1.0.2:8000
Sent Image frame: 1 subframe: 1 packets: 424 bytes: 2112368 to 10.1.0.2:8000
Sent Reset frame: 1 subframe: 0 packets: 424 bytes: 2112368 to 10.1.0.2:8001
Sent Reset frame: 1 subframe: 1 packets: 424 bytes: 2112368 to 10.1.0.2:8001
frame: 2
Sent Image frame: 2 subframe: 0 packets: 424 bytes: 2112368 to 10.1.0.3:8000
Sent Image frame: 2 subframe: 1 packets: 424 bytes: 2112368 to 10.1.0.3:8000
Sent Reset frame: 2 subframe: 0 packets: 424 bytes: 2112368 to 10.1.0.3:8001
Sent Reset frame: 2 subframe: 1 packets: 424 bytes: 2112368 to 10.1.0.3:8001
3 frames completed, 25348416 bytes sent in 0.601 secs