4. Extending PEPI¶
Generally, you’ll want to extend PEPI by creating new servers and adding support for new cameras. The client-side software should work with any server out-of-the-box, but there are still some features that need to be added to the client.
4.1. PEPI Theory¶
PEPI can be divided into the client-side and the server-side. As discussed, the client-side doesn’t really need to be extended–the server is where the interesting extensions can happen.
Servers can be divided into two components:
- The actual server itself (the
CameraServer
) - Cameras connected to the server (the
Camera
)
We do not mandate that a CameraServer
must take Camera
objects, but it is strongly recommended unless you have a valid reason (e.g. very complicated hardware requirements). The provided Python implementations shows how this may be implemented.
4.1.1. Languages¶
PEPI is indifferent to which language you implement your server in, so long as it can be accessed over Apache Thrift. Thrift’s language bindings include:
- C++
- C#
- Cocoa
- D
- Delphi
- Erlang
- Haskell
- Java
- OCaml
- Perl
- PHP
- Python
- Ruby
- Smalltalk
- ..plus others in the works or supported by third parties
Therefore, any of the above languages can be used to implement PEPI server-side components.
4.2. Writing New Servers¶
4.2.1. Interface Definition File¶
At the heart of PEPI is its interface definition file pepi.thrift
. This defines the interface used to access servers and therefore specifies what functions you need to implement.
/*******************************************************************************
* File: pepi.thrift
* Author: Curtis West
* -----------------------------
* Interface definition file for Apache Thrift.
******************************************************************************/
/*******************************************************************************
Thrown when a requested image is not available on the server
*******************************************************************************/
exception ImageUnavailable {
1: string message,
}
/*******************************************************************************
A Camera provides images from a physical camera in the form of RGB arrays.
*******************************************************************************/
service Camera {
/* still
* description: returns a still image capture from the camera at the currently
* set resolution
* returns: multidimensional array of row, column, RGB representing the image
*/
list<list<list<i16>>> still()
/* low_res_still
* description: gets a 640 x 480px still from this camera for previewing
* returns: multidimensional array of row, column, RGB representing the image
*/
list<list<list<i16>>> low_res_still()
/* still
* description: gets the maximum resolution supported by this camera
* returns: a list of length 2 representing the resolution i.e. (x, y)
*/
list<i16> get_max_resolution(),
/* get_current_resolution
* description: gets the current resolution of this camera
* returns: a list of length 2 representing the resolution i.e. (x, y)
*/
list<i16> get_current_resolution(),
/* set_resolution
* description: if supported, sets the resolution of the camera
*/
oneway void set_resolution(1:i16 x, 2:i16 y)
}
/*******************************************************************************
A CameraServer serves as a wrapper around a camera and provides a number of
utility functions for managing the server and camera.
*******************************************************************************/
service CameraServer {
/* ping
* description: Used to ping the server.
* returns: True, always
*/
bool ping(),
/* identify
* description: Gets this server's unique identifier.
* returns: String containing this server's identifier
*/
string identify(),
/* stream_urls
* description: Gets a list of URL where the stream of this server's cameras
may be accessed, if they exist.
* returns: List of strings containing URLs, or an empty list.
*/
list<string> stream_urls(),
/* ping
* description: Shuts down this server. This does not need to be implemented,
* but the server must accept the function call.
*/
oneway void shutdown(),
/* start_capture
* description: Captures a still from this server's camera(s) and stores in
internally under the given `data_code` for later retrieval.
*/
oneway void start_capture(1:string data_code),
/* retrieve_stills_png
* description: Retrieves .png images that were captured using `start_capture`
under the specified `data_code` (if they exist), encoded as
PNGs.
* throws: ImageUnavailable
* returns: a list of strings, where each string contains one image encoded as
a PNG file. Each string should be able to be dumped directly to
the disk and still form a valid PNG file.
*/
list<string> retrieve_stills_png(1:string with_data_code) throws(1:ImageUnavailable unavailable),
/* retrieve_stills_jpg
* description: Retrieves images that were captured using `start_capture`
under the specified `data_code` (if they exist), encoded as
JPEGs.
* throws: ImageUnavailable
* returns: a list of strings, where each string contains one image encoded as
a JPEG file. Each string should be able to be dumped directly to
the disk and still form a valid JPEG file.
*/
list<string> retrieve_stills_jpg(1:string with_data_code) throws(1:ImageUnavailable unavailable),
/* enumerate_methods
* description: Returns a dictionary of the methods supported by this server.
This is currently not used for any function as of v3,
so you may choose to just return an empty dictionary,
but be aware that this may change in the future versions.
* returns: A dictionary with:
key: method name
value: a list of argument names that the method takes
*/
map<string, list<string>> enumerate_methods()
}
Depending on the language you choose to implement your new server/camera, the exact format of how you implement these functions will vary, but generally you’ll just write the functions exactly as listed (but in the syntax of your language).
From the perspective of writing a server implementation, there are no special requirements from Thrift; you don’t need to return Thrift types or use Thrift objects. Your server won’t even know its been called from Thrift (sometimes it won’t be). Instead, treat component implementations as handlers that are called when in response to a Thrift requests, with Thrift managing all the necessary type conversions and network transports.
4.2.2. Python’s BaseCameraServer¶
PEPI provides a minimal implementation of a CameraServer under the class BaseCameraServer
.
In most cases, BaseCameraServer
can be used without modification as long as you can provide the Camera
you’d like to use. However, you may wish to override some methods to better suit your use case. For example, RaspPiCameraServer
overrides the identifier()
method to use the Raspberry Pi’s CPU serial number as the ID.
If your server is being implemented in another language, it is still beneficial to refer to this implementation to understand how certain operations are accomplished.
class BaseCameraServer(object):
"""
BaseCameraServer is the minimal Python implementation of a CameraServer
as defined in ``pepi.thrift``. CameraServers are used in with the
Apache Thrift protocol to provide RPC mechanisms, allowing control
of this server over RPC, if it is launched with Thrift.
A CameraServer subclassing BaseCameraServer may override any of these
methods to better reflect their use. However, care must be taken to
ensure that the side effects of the subclass's methods do not affect
other methods. For example, if you were to change the capture method
to store images in a list for whatever reason, you would need to change
the image retrieval methods.
A CameraServer's use-case is to provide a server that controls a
number of cameras to be controlled in a consistent manner. This allows
for a client to seamlessly control all implementations of CameraServer's,
over Thrift without needing to concern themselves with what cameras are
attached, the procedure call names, etc.
This BaseCameraServer implementation supports multiple connected cameras,
that are transparent to the connecting client. When retrieving images, a
list of encoded images are returned. The order of this list remains
consistent across procedure calls.
"""
StreamInfo = collections.namedtuple('StreamInfo', 'port, folder, streamer')
STREAM_PORT = 6001
def __init__(self, cameras, stream=True):
# type: ([AbstractCamera], bool) -> None
"""
Initialises the BaseCameraServer.
:param cameras: a list of AbstractCamera objects
:param stream: True to start streams for all cameras, False to not.
"""
# self.cameras = CameraManager(cameras)
self.cameras = cameras
self._stored_captures = dict()
self.streams = dict()
self.identifier = str(uuid.uuid4().hex)
if stream:
StreamInfo = collections.namedtuple('StreamInfo', 'port, folder, streamer, capturer')
for count, camera in enumerate(cameras):
port_ = self.STREAM_PORT + count
folder_ = tempfile.mkdtemp()
streamer_ = MJPGStreamer(folder_, port=port_)
capturer = CameraTimelapser(camera=camera, folder=folder_, interval=0.5)
capturer.start()
self.streams[camera] = StreamInfo(port=port_, folder=folder_, streamer=streamer_, capturer=capturer)
def cleanup(): # pragma: no cover
"""
Cleans up after this server by destroying connected cameras
and their streams, and erasing the stored images.
"""
logging.info('Cleaning up RaspPiCameraServer')
self._stored_captures = None
self.cameras = None
self.streams = None
logging.info('Cleanup complete for RaspPiCameraServer')
atexit.register(cleanup)
def ping(self):
# type: () -> bool
"""
Ping the server to check if it is active and responding.
:return: True (always)
"""
logging.info('ping()')
return True
def identify(self):
# type: () -> str
"""
Get the unique identifier of this server.
:return: the server's unique identifier string
"""
logging.info('identify()')
return self.identifier
@staticmethod
def _current_ip():
# type: () -> str
return IPTools.current_ips()[0]
def stream_urls(self):
# type: () -> [str]
"""
Get the a list of URLs where the MJPG image stream of each camera
connected to this server may be accessed.
The order of the returned images is consistent, e.g. Camera #1, #2
.., #x returned in that order.
:return: a list of the stream URLs as a string
"""
logging.info('stream_urls()')
out_urls = []
for _, stream_info in viewitems(self.streams):
out_urls.append('http://{}:{}/stream.mjpeg'.format(self._current_ip(), stream_info.port))
return out_urls
def shutdown(self):
# type: () -> None
"""
Shutdown the server (i.e. power-off).
Subclasses may choose to
ignore calls to this function, in which case they should override
this function to do nothing.
:return: None
"""
logging.info('shutdown()')
os.system('shutdown now')
def start_capture(self, data_code):
# type: (str) -> None
"""
Immediately starts the process of capturing from this server's Camera(s),
and stores the captured data under the given unique data_code.
Note: the received `data_code` is assumed to be unique. Subclasses may
choose to implement better isolation methods, but this is not
guaranteed nor required.
:param data_code: the requested data_code to store the capture under
:return: None
"""
logging.info('start_capture(data_code: {})'.format(data_code))
captures = []
# TODO: parallelize capture from all cameras
for camera in self.cameras:
try:
captures.append(Image.fromarray(np.array(camera.still(), dtype=np.uint8)))
except (AttributeError, TypeError, ValueError) as e:
logging.warn('Could not construct image from received RGB array: {}'.format(e))
continue
if captures:
self._stored_captures[data_code] = captures
logging.info('Stored_captured after start_capture(): {}'.format(self._stored_captures.keys()))
def _retrieve_and_encode_from_stored_captures(self, data_code, encoding, quality):
# type: (str, str, int) -> [str]
try:
image_list = self._stored_captures.pop(data_code)
except KeyError:
raise ImageUnavailable('No images are stored under the data_code "{}"'.format(data_code))
else:
out_strings = []
for image in image_list:
image_buffer = BytesIO()
image.save(image_buffer, encoding, quality=quality)
out_strings.append(image_buffer.getvalue())
return out_strings
def retrieve_stills_png(self, with_data_code):
# type: (str) -> [str]
"""
Retrieves the images stored under `with_data_code`, if they exist, and
encodes them into a .png str (i.e. bytes).
The order of the returned images is consistent, e.g. Camera #1, #2
.., #x returned in that order.
:param with_data_code: the data_code from which the image will be retrieved
:raises: ImageUnavailable: when image requested with an invalid/unknown data_code
:return: a list of strings with each string containing an encoded as a .png
"""
logging.info('retrieve_stills_png(with_data_code: {})'.format(with_data_code))
return self._retrieve_and_encode_from_stored_captures(with_data_code, 'PNG', quality=3)
def retrieve_stills_jpg(self, with_data_code):
# type: (str) -> [str]
"""
Retrieves the images stored under `with_data_code`, if they exist, and
encodes them into a .jpg str (i.e. bytes).
The order of the returned images is consistent, e.g. Camera #1, #2
.., #x returned in that order.
:param with_data_code: the data_code from which the image will be retrieved
:raises: ImageUnavailable: when image requested with an invalid/unknown data_code
:return: a list of strings with each string containing an encoded as a .jpg
"""
logging.info('retrieve_stills_jpg(with_data_code: {})'.format(with_data_code))
return self._retrieve_and_encode_from_stored_captures(with_data_code, 'JPEG', quality=85)
def enumerate_methods(self):
# type: () -> [(str, str)]
"""
Retrieves a map of the methods available on this server. This is useful
for clients to verify the methods it can expect to be able to call
if being called remotely.
:return: dict of <method_name: [arguments]>
"""
import inspect
methods = inspect.getmembers(self, predicate=inspect.ismethod)
output_dict = dict()
for _tuple in methods:
name, pointer = _tuple
args = inspect.getargspec(pointer).args
try:
args.remove('self')
except ValueError: # pragma: no cover
pass
output_dict[name] = args
return output_dict
4.3. Writing New Cameras¶
If you choose to use a Camera
object with your server, then you should implement your camera according to the Camera
interface.
In Python, AbstractCamera
is provided as an abstract class with some implemented methods. It is an abstract class rather than a base class (like CameraServers
) because it’s impossible to cater for all possible connected hardware.
The most important (and tricky) method in a Camera
is it’s still()
method that returns a multi-dimensional array of 0-255 RGB pixels (row, column, RGB). For example, MyConcreteCamera
is implemented in Python and captures RGB images at a 4-by-3 pixel resolution:
>>> camera = MyConcreteCamera()
>>> image = camera.still()
>>> print(type(image))
<type 'numpy.ndarray'>
>>> print(image.shape)
(3, 4, 3)
>>> image
array([[[244, 213, 53], # Row 1, Col 1
[141, 130, 195], # Col 2
[229, 156, 94], # Col 3
[204, 19, 191]], # Col 4
[[105, 202, 239], # Row 2, Col 1
[183, 109, 243], # Col 2
[164, 190, 1], # Col 3
[216, 191, 63]], # Col 4
[[160, 232, 240], # Row 3, Col 1
[ 86, 186, 252], # Col 2
[ 19, 212, 221], # Col 3
[253, 143, 29]]], dtype=uint8) # Col 4
Most physical cameras don’t provide a RGB array. The easiest way to transform from JPG or PNG (preferred) files is to use a library such as Pillow (previously, PIL). In Python, we provide a utility class server.abstract_camera.RGBImage
based on Pillow that can do some of these conversions for you.
from server import RGBImage
import numpy as np
class MyConcreteCamera(AbstractCamera):
def __init__(self):
self._camera = MyDSLRCamera()
def still(self):
png = self._camera.get_png()
return np.array(RGBImage.fromstring(png))
Alternatively, if you wish to use Pillow directly:
from io import BytesIO
from PIL import Image
import numpy as np
class MyConcreteCamera(AbstractCamera):
def __init__(self):
self._camera = MyDSLRCamera()
def still(self):
png_buffer = BytesIO()
png_buffer.write(self._camera.get_png())
png_buffer.seek(0)
image = Image.open(png_buffer)
return np.array(Image.open(png_buffer))
4.3.1. Testing Your Camera Implementation¶
PEPI includes tests that you can run against your new camera implementation to see if it returns the correct values both natively in Python and over Apache Thrift. Note that this isn’t an exhaustive test of your camera implementation and how it handles errors etc., but instead just a test to check the correct values are returned.
To setup these tests against your server, you’ll need to define a few pytest fixtures that is used to “inject” your camera into the tests:
import pytest
from server.tests import AbstractCameraContract, AbstractCameraOverThrift
import MyConcreteCamera
class TestMyConcreteCamera(AbstractCameraContract):
@pytest.fixture(scope="module")
def camera(self):
return MyConcreteCamera()
class TestDummyCameraOverThrift(AbstractCameraOverThrift):
@pytest.fixture(scope="module")
def local_camera(self):
return MyConcreteCamera()
Refer to the Testing section for more details on testing in PEPI.
4.4. Raspi Server Implementation¶
A subclass of BaseCameraServer
is provided with PEPI for use with Raspberry Pi’s, RaspPiCameraServer
. This serves as a useful example on how to extend a language’s base implementation to customize functionality.
class RaspPiCameraServer(BaseCameraServer):
"""
An implementation of a BaseCameraServer for a Raspberry Pi.
"""
def __init__(self, cameras, stream=True):
super(RaspPiCameraServer, self).__init__(cameras, stream)
# Set identifier based on the RPi's CPU serial number
try:
with open('/proc/cpuinfo', 'r') as f:
for line in f:
if line[0:6] == 'Serial':
self.identifier = line[10:26]
except IOError:
pass
As we have previously discussed, in most cases the procedures implemented in BaseCameraServer
can be used for new servers. Simply subclass BaseCameraServer
and your new server will inherit all of these implementations, which have been thoroughly tested and refined.
RaspPiCameraServer
does exactly this: subclasses BaseCameraServer
and overrides the identify()
procedure to better suit its use case by using the CPU serial number to identify the server (to allow rapid deployment without manual setup on each server). This demonstrates just how easy extendnig servers are (at least in Python, but generally you’ll only need one base server to extend written for each language).