Mark N R Smith, Marwan Shaker, Shigang Yue and Tom Duckett, "a Thin Middleware for Simulated Robot Vision Applications," ICCSIT International Conference on Computer Science and Information Technology, Chengdu, China, 2011
altURI is middleware software for use with a robot simulator designed to be used in vision and robotics research at the University of Lincoln and elsewhere. The software provides modules to receive images and sensor information from and send commands to virtual robots within a USARSim environment.
The software provides simple and configurable interfaces to any number of simulated robots using a combination of web and Application Programmer Interface mechanisms.
This allows simulated robots to be controlled by a wide range of application software such as MATLAB, C++ and Java. All components of the altURI system are licensed as Open Source.
The USARSim (Unified System for Automation and Robot Simulation) system was created to provide a realistic low cost simulation tool for robots in real environments. It is based on the commercial games platform Unreal Tournament (Epic Games, 2004, 2009) which is customised to provide models of various robots and environments. Additional software components are provided to support image and sensor data acquisition and implement robot actions.
A high performance vision based middleware for robot simulation, with maximal simulator decoupling and running in an easily available environment.
1. To provide modules that are fast, flexible and easily available to a wide range of robot control application development platforms.
2. To support image acquisition, robot commands and sensor output for development applications that works with USARSim using a superset of simulator interfaces.
The altURI software contains two components and two support programs. The two components are the vision acquisition module and the command/sensor module.
The vision acquisition module obtains images from the simulator game graphics engine and makes them available to robot control applications. Images are supplied at the same resolution as the simulator but smaller or partial images (multi-view) are available. A command-line support program to start the simulator and load the vision component into a graphics process is provided.
A command/sensor module instance is specific to a robot type (model) and controls a single robot in the simulator. The commands that can be sent, and the sensor values that can be received are specified in a configuration file.
The other support component is a test application which loads both modules and can display images retrieved and validate the control and sensor configurations.
During research work the USARSim robot simulator was proposed for developing a robot control application during 2009. However the USARSim project was perceived (in autumn 2009) to have some disadvantages:
1. The system was moving to Unreal Tournament 3 (from UT2004) and all the required new modules were not yet in place;
2. A control system interface to general software applications was not available and the existing control applications appeared to be fairly specialised;
3. The image acquisition system used in UT2004 did not appear to work well;
4. The overall future of the USARSim system was not clear
The altURI software was designed to control robots in the USARSim robot simulator running on the Unreal Tournament game. It consists of three components plus a test harness. Two of the components are Windows Dynamic Link Libraries (DLLs); one sends commands and the other acquires images from the USARSim game environment. A small utility program is provided to load the image acquisition DLL and start the USARSim game program. A test harness is provided that can be used to view images and sensor data as well as send control commands using buttons on the test program.
The design goals were met by implementing the altURI modules as Windows Dynamic Link Libraries. The image processing module can supply images from 32-bit and 64-bit graphics applications (i.e. simulators) using DirectX8, DirectX9, DirectX10, DirectX11 or OpenGL graphics for any Windows operating system.
Images can be supplied as OpenCV, USARSim FrameData and Matlab (using the control/sensor module) formats via shared memory or as web images using HTTP requests. The vision component web server can supply images in PNG, JPG or TIFF formats and display a continuous 'movie' of the simulator view.
The altURI modules support a superset of simulator interfaces by working with most graphics engines and by implementing a TCP/IP socket system that can send configured commands with associated numerical values, and retrieve sensor values by parsing returned sensor message text.
The four components of the system altURI are:
altURI_Hook.dll – a 32-bit Windows DLL that loads into a graphics environment such as UT2004, UT3. The hook DLL provides a cut down HTTP web server to distribute images. It also provides an Application Programmer Interface (API) similar to the original USARSim hook module and similar to the UPIS system.
altURI_LoadDLL.exe – a 32-bit Windows command line program that loads altURI_Hook.dll into the environment of a graphics program.
altURI_Cmd.dll – a 32-bit Windows DLL that provides an Application Programmer Interface (API) to functions that sends commands to the USARSim game environment and receives remote images via HTTP.
altURI_UI.exe – a 32-bit Windows test program that uses altURI_Hook.dll and altURI_Cmd.dll to receive images and sensor data and send commands. Allow testing of robot commands and sensor information configuration.
Figure 1 - altURI Components
Figure 2 - altURI Test Application showing 640x480 pixel image USARSim UT2004 TallTestWorld from AirRobot Camera.
Figure 3 - altURI Test Application showing 800x600 pixel image: USARSim UT3 Conveyor map with P3AT Robot.
How to Use altURI
The manual as a pdf is here
The run time (non-developer x86) components are available here
The run time (non-developer x64) components are available (TBA - you can compile it yourself with the source below)
The full source code is here
Installation (non-developer version)
1. Install Unreal Tournament 2004 or Unreal Tournament 3 from CD or via Steam
2. Install USARSim for the version of Unreal Tournament you installed in step 1.
3. Check you have a map you want to use in the correct folder for USARSim.
4. To make images available use the altURI_LoadDLL batch file to load altURI_Hook into the game (edit the paths and map name to match your file locations and names)
5. Check you can see images a USARSim image in your web browser.
Please contact the developer: Mark N R Smith ( mnsmith a-t lincoln dot ac dot uk ) for further information.
altURI is released under a BSD OpenSource license:
Copyright (c) 2010, Mark N R Smith, All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
Redistributions of source code must retain the above copyright notice, this list
of conditions and the following disclaimer. Redistributions in binary form must
reproduce the above copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials provided with the
distribution. Neither the name of the author nor the names of its contributors
may be used to endorse or promote products derived from this software without
specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
altURI uses code from Taksi that is licensed under the BSD OpenSource License.
altURI also uses code from, OpenCVX that is licensed under the MIT OpenSource license