`st3m.ui.widgets` module

Basic Usage

flow3r widgets are objects that take raw user input and transform it in some or the other useful data. While this term is often strongly associated with a visual output, we use it in a more generic sense: The output of a widget doesn’t necessarily go to the screen, but may be routed wherever.

class Widget

think(ins: InputState, delta_ms: int) → None: Takes input from the hardware drivers and updates the state of the widget accordingly.

on_exit() → None: Suspends the widget. Should be called when .think() will not be executed anymore for a while to make sure no stale data is used when the widget is reactivated later. This also allows the widget to deallocate resources it might be using. Always call when application is exited.

on_enter() → None: Prepares/unsuspends the widget for immediate impending use. This may allocate resources needed for data processing etc.

The typical pattern for using widgets can be very simple. Since in many cases all the methods above are applied to all widgets at the same time, it is convenient to keep a list of all widgets, but for easily readable attribute access it makes also sense to keep a seperate named reference for each widget like so:

from st3m.ui import widgets

class App(Application):
    def __init__(self, app_ctx):
        super().__init__(app_ctx)
        # keep direct references around for easy access...
        self.some_widget = widgets.SomeWidget()
        self.other_widget = widgets.OtherWidget()
        # ...but also put them in a list for batch processing
        self.widgets = [self.some_widget, self.other_widget]

    def on_enter(self, vm):
        super().on_enter(vm)
        for widget in self.widgets:
            widget.on_enter()

    def on_exit(self):
        super().on_exit()
        for widget in self.widgets:
            widget.on_exit()

    def think(self, ins, delta_ms):
        super().think(ins, delta_ms)
        for widget in self.widgets:
            widget.think(ins, delta_ms)

        some_value = self.some_widget.attribute

If you forget to call on_exit()/on_enter() you might not notice during testing but get strange bugs when people exit and reenter your application - it’s best to always keep a list of all your widgets around so that you don’t forget these method calls,

If you don’t need all your widgets all the time and want to save CPU by not calling .think() on all of them, it is also advised to use on_exit()/on_enter() as appropriate to avoid unwanted side effects.

The primary motivation of widgets is to provide captouch interaction processing, but these APIs are a bit more complicated, so we start with the simpler group of sensor widgets:

Sensor Widgets

These widgets provide common mathematical transformations and filtering of sensor data. They are designed to easily enhance user interfaces with motion control and similar interactions.

class Altimeter(Widget)

__init__(ref_temperature: Optional(float) = 15, filter_stages: int = 4, filter_coeff: float = 0.7) → None: Initializes a pressure-based altimeter. ref_temperature initializes the attribute of the same name. Since the meters_above_sea output is very noisy a multistage pole filter is provided. The number of stages is set by filter_stages and must be 0 or greater. filter_coeff initializes the attribute of the same name.

meters_above_sea: float: Rough estimate of altitude in meters. The “above sea” bit is mostly theoretical, see .temperature, it is best used as a relative measurement. Very sensitive to pretty much everything, like moving your arms, closing windows and doors, wind, you name it. Heavy filtering is recommended, meaning for practical purposes you may expect a very slow response time in the magnitude of >10s.

ref_temperature: Optional(float): Set to None to use the built-in temperature sensor for calculations, else use the value supplied here - wait, why wouldn’t you use the built-in sensor? The reason for this that temperature is just a proxy of the density curve of the air column, from your position all the way up into space! This total weight is the main source of local pressure, so we can use it to estimate how much air column there is on top of you, i.e. how high up in the athmosphere you are. So if you’re in a heated room, your local temperature is misrepresentative of most of that column, and as soon as you step outside (or in a differently heated room) your reading will jump all over the place. Using the integrated sensor therefore doesn’t really make the data better unless you are outside - it just makes it more jumpy.

Pressure between rooms is fairly consistent, albeit noisy. Say we don’t care for the absolute height, but how height changes over time as we move flow3r up and down: Using a constant temperature guesstimate at worst introduces a minor scaling error but removes all jumps from local temperature changes. It is almost always the better option to ignore it.

If you do want to perform actual absolute altimetry we advise polling weather services for local temperature data. Also barometric pressure has other sources of error that you might take into account. It’s probably best if widgets don’t silently connect to the web to poll data, so this will never be a fully automatic feature.

filter_coeff: float: Cutoff coefficient for all filter stages. Should typically be between 0 and 1. Each stages is updated once per .think() by the rule output = input * (1-filter_coeff) + output * filter_coeff, so that higher filter_coeff values result in slower response. Changing this parameter while the widget is running will not result in instant response as the change needs to propagate through the filter first, if you need two different response behaviors at the same time it is best to create two seperate instances or filter externally.

Note: This filter is not synchronized to actual data refresh rate but rather to the think rate, meaning that the same sample may occur in the buffer multiple times in succession or that data points might be skipped, and that response time is a function of think rate. The actual data refresh rate is >20ms.

class Inclinometer(Widget)

__init__(buffer_len: int = 2) → None:: Initializes a inclinometer. Noise can be removed with a windowed averaging filter, buffer_len sets the maximum window length of that filter. At runtime you may switch between filter lengths with the filter_len attribute as you wish, but its maximum must be set at initialization. Must be greater than 0.

inclination: float: Polar coordinates: How much flow3r is tilted compared to lying face-up on a table, in radians. Facing up is represented by a value of 0, facing down by π, sideways is π/2 et cetera.

Range: [0..π]

azimuth: float: Polar coordinates: Informally this parameter represents which petal is pointing up most: Say flow3r is upright (.inclination = π/2), a value of 0 means the USB-C port is pointing up. A positive value indicates that it is rotated clockwise from there, as a steering wheel analogy you’re “turning right”. Of course this value works just the same for all values of inclination, but there is a caveat: If fully facing up or facing down, so that no petal is pointing up more than the others, this parameter will jump around with the random noise it picks up. If this causes you issues it’s best to check inclination whether it is tilted enough for your purposes.

Range: [-π..π]

roll: float: Aviation coordinates: How far flow3r is tilted sideways to the right, in radians.

Range: [-π..π]

See coordinate system note at the pitch attribute description.

pitch: float: Aviation coordinates: How far flow3r is tilted backwards, in radians.

Range: [-π..π]

Coordinate system note: Isn’t yaw missing here? Yes, but we can’t obtain it from the gravity vector, so we’re skipping it. Also you might notice that normally the pitch angle only covers a range of π, this implementation however spans the full circle.

This has a side effect: Normally, the pitch angle range is limited to π, but here we go full circle. We do this to avoid a discontinuity that would normally jump between interpreting, say, a unrolled pitch-down to a half-rolled pitch-up. The price we pay for this is that our modified coordinate system fails when facing down: With full range on both angles it is undecidable whether these positions come from rolling or pitching by an angle of π. We distribute it based on azimuth instead, which tends to be jumpy in that area, so both pitch and roll experience full-range random jumps in that zone.

filter_len: int: Sets the length of the windowed averaging filter. Copies the value buffer_len at initialization. Must be greater than zero and smaller or equal to buffer_len. This parameter only affects the values of the output attributes and doesn’t change any of the available data, so you may even change it several times within the same think cycle and retrieve the same output attributes with different response times if needed.

Note: This filter is not synchronized to actual data refresh rate but rather to the think rate, meaning that the same sample may occur in the buffer multiple times in succession or that data points might be skipped, and that response time is a function of think rate. The actual data refresh rate is >10ms.

Captouch Widgets

The raw output data of the captouch driver is quite rough around the edges as described in the captouch module documentation. The widgets provided here aim to provide an intermediate layer to remove all the pitfalls that this poses; you might think that many of these are trivial and barely deserving of their own classes, but the devil is in the details. They are not without quirks, but if we ever figure out how to fix them these updates will directly benefit your application, and there is value in having consistent quirks across the entire userland if possible.

These widgets often don’t work with the default captouch driver configuration. To make it easy to generate a configuration that works for all widgets used at a given time, each widget can add its requirements to a captouch.Config object. Typically this is done at initialization:

from st3m.ui import widgets
import captouch

class App(Application):
    def __init__(self, app_ctx):
        super().__init__(app_ctx)
        # create a default configuration
        self.captouch_config = captouch.Config.default()
        # create some widgets and add requirements to the config above
        self.slider = widgets.Slider(self.captouch_config, 4)
        self.scroller = widgets.Scroller(self.captouch_config, 2)

        self.widgets = [self.slider, self.scroller]

    def on_enter(self, vm):
        super().on_enter(vm)
        # apply the configuration when entering
        self.captouch_config.apply()
        for widget in self.widgets:
            widget.on_enter()

    # (other methods same as in the general Widget example above)

The example above is a bit wasteful: The .default() configuration activates a lot of data channels that we might not be using and which slow down the data rates of the channels that we actually care about. In the example above, the Scroller widget would benefit a lot from the 3x-ish increase in data rate that starting with an .empty() configuration would yield (see the captouch module documentation for details).

However, not all captouch data access is happening via widgets; an application might use edges from st3m.input.InputController or primitive position output from st3m.input.InputState. When starting with a .default() configuration and only adding widget requirements these are guaranteed to work; however, for best performance it is ideal to start with an .empty() one and add them manually as we will demonstrate below.

One more case would be that not all widgets are in use at the same time; in that case, it makes sense to create different configs to switch between. We can only pass one configuration to the initializer, but that just forwards it to the .add_to_config() method, which we can call as many times with different config objects as we like. The initializer must still receive a valid config to serve as a beginner’s trap safety net.

Here’s a slightly more advanced example:

from st3m.ui import widgets
import captouch

class App(Application):
    def __init__(self, app_ctx):
        super().__init__(app_ctx)
        self.captouch_configs = [None, None]
        for x in range(2):
            # create a empty configuration
            captouch_config = captouch.Config.empty()
            # manually add petal 5 in button mode
            captouch_config[5].mode = 1
            self.captouch_configs[x] = captouch_config

        # the slider widget is only sometimes in use:
        self.slider_enabled = True

        # add both widget requirements to configs[0]
        self.slider = widgets.Slider(self.captouch_configs[0], 4)
        self.scroller = widgets.Scroller(self.captouch_configs[0], 2)

        # only add the scroller configs[1]:
        self.scroller.add_to_config(self.captouch_configs[1])

        self.widgets = [self.slider, self.scroller]

    def on_enter(self, vm):
        super().on_enter(vm)
        # select config we want
        config_index = 0 if self.slider_active else 1
        self.captouch_configs[config_index].apply()
        for widget in self.widgets:
            if self.slider_enabled or widget != self.Slider:
                widget.on_enter()

    def on_exit(self):
        super().on_exit()
        # calling widget.on_exit() multiple times without pairing it with on_enter()
        # is okay so we don't need to check here
        for widget in self.widgets:
            widget.on_exit()

    def think(self, ins, delta_ms):
        super().think(ins, delta_ms)
        for widget in self.widgets:
            widget.think(ins, delta_ms)

        if self.input.captouch.petals[5].whole.pressed:
            self.slider_enabled = not self.slider_enabled
            if self.slider_enabled:
                self.slider.on_enter()
            else:
                self.slider.on_exit()
            config_index = 0 if self.slider_active else 1
            self.captouch_configs[config_index].apply()

        if self.slider_enabled:
            do_thing_with_slider_value(self.slider.pos)
        do_thing_with_scroller_value(scroller_value = self.scroller.pos)

With that out of the way, let’s finally look at the base class of all captouch widgets:

class CaptouchWidget(Widget)

__init__(config: CaptouchConfig, gain: complex = 1, constraint: Constraint = None, friction: float = 0.7, bounce: float = 1) → None: Initializes widget and adds its requirements to config via .add_to_config(). This is mandatory as most widgets do not work with the default captouch driver configuration. The other arguments initialize the parameters of the same names.

add_to_config(config: CaptouchConfig) → None: Adds the requirements of the widget to a captouch config object, see examples above.

pos: complex: Main data output of widget. Individual behavior is documented in the subclasses below. Default value is 0 unless specified otherwise.

While this is primary intended for read access, writing is allowed. This is useful for example for initializing a Slider to a default value or resetting a widget that does relative movements. Writing does not automatically apply the current .constraint but you can do so manually as with the “bad bounce” workaround (see Constraints section for more details).

active: bool: Read only: Whether the widget is currently interacted with or not. Updated by .think().

gain: complex: Multiplier that scales how much a physical captouch interaction changes the .pos output. Since it is complex-valued this can also include a rotation around the origin, for example when cmath.rect(scaling_factor, angle) or scaling_factor * captouch.PETAL_ROTORS[petal_index] is used.

constraint: Constraint: Limits the possible values of .pos after .gain is applied. Note that swapping or modifying constraint after widget initialization may result in un-“physical” side effects, see documentation of the Constraint class below. Multiple widgets may use the same constraint, this results in the same behavior as individual identical constraints.

friction: float: How fast the speed of pos decays if it is moving freely. Must be positive or zero. Not used by all widgets.

bounce: float: By how much the absolute speed is multiplied when colliding with a constraint wall. Must be positive or zero. Not used by all widgets.

Single-Petal Widgets

class PetalWidget(CaptouchWidget)

Parent class of a widget that reacts to a single petal.

__init__(config: CaptouchConfig, petal: int, **kwargs) → None: Initializes widget and ties it to the petal with the index specified by the petal argument. config and **kwargs are forwarded to the initializer of CaptouchWidget.

class Slider(PetalWidget)

This widget allows users to set an application parameter with absolute touch position. If no constraint argument is passed to the initializer it defaults to a unit circle constraint. Ignores friction and bounce parameters.

pos: complex: Absolute position of the last touch.

class Scroller(PetalWidget)

This widget allows users to change an application parameter with relative touch position. If the friction is set to 1, this can be used as an “incremental” Slider, friction values between 0 and 1 allow for crude swipe gestures. Note: due to captouch data limitations it is very hard to not swipe at least a little bit, expect some residual drift when planning a UI.

If no constraint argument is passed or it is None to the initializer it throws a TypeError since else pos might grow without bounds up into the NaN range which would cause harder-to-detect rounding issues and crashes.

pos: complex: Relative position that is changed incrementally with touch.

Multi-Petal Widgets

class MultiSlider(CaptouchWidget)

If no constraint argument is passed to the initializer this defaults to a unit circle constraint. Ignores friction and bounce parameters.

pos: complex: Takes all top petals as one big surface. Active if only one touch is registered on that surface (i.e., if at most 2 adjacent petals are pressed). Normalized so that the absolute value is typically between 0.5 and 1. The center values cannot be reached, but it is initialized to 0 anyways.

Constraints

Constraints limit the pos output of a CaptouchWidget to a region. This needn’t be simple value clipping but can also include more advanced behavior such as overflowing or bouncing off walls. Since you probably want different behavior depending on whether you’re “holding” the “ball” or not, and the asynchronous nature of the petal log may result in multiple states per think, these needed to be integrated deeply into the widgets. They’re on this weird state of complexity where it feels tempting to either go full game engine (which would however be hard to justify effort) or radically simplifying them (which would however restrict some use cases).

We decided to keep them on this middle ground for now, which has some implications:

Resizing/centering/rotating them during runtime may result in “bad bounces” if a widget is at a position that was previously within the constraint but now is outside. As a mitigation call .apply_hold() manually to avoid such a bounce.
Constraints can’t give positional outputs a physical “shapeness” to collide with others asthey have no concept of mass or motion of self.
There is no computationally efficient way to combine them to create a “game level”. If you aim for this it is best to make one “giant constraint” which implements all the math from scratch.

class Constraint

__init__(size: [float, complex] = 1, center: complex = 0j, rotation: float = 0)

Creates a new constraint located at the given center with given rotation. The parameters initialize the attributes of the same name.

Note: Not all constraint subclasses must accept these parameters, we merely grouped them up here since all constraints we provide use them. If you implement your own application specific constraint feel free to use arbitrary arguments for the initializer.

apply_hold(pos: complex) → complex: Returns pos limited to whatever range the constraint defines. Typically called automatically by the widget(s) using the constraint, manual calls are rarely needed.

apply_free(pos: complex, vel: complex, bounce: float): Returns (pos, vel) tuple. Like apply_hold, but allows for bouncing off of walls. Typically called automatically by the widget(s) using the constraint, manual calls are rarely needed. Not all widgets call this method; at this point in time, only the ones that use friction do so. If you inherit from this base class and do not define it, it defaults to limiting pos with .apply_hold() and leaving vel untouched.

size: complex

Linearily scales the size of the constraint. The real component must be greater than 0, the imaginary component must be no less than 0, else setting will raise a ValueError. Setting it with a float or a complex number whose imaginary component is 0 results in a “square-shaped” value, or size = complex(size.real, size.real).

Note: Not all constraint subclasses must have this attribute, see __init__ note. Changing this attribute at runtime may result in “bad bounces”, see above.

center: complex

Shifts the center of the constraint.

Note: Not all constraint subclasses must have this attribute, see __init__ note. Changing this attribute at runtime may result in “bad bounces”, see above.

rotation: float

Rotates the constraint in radians.

Note: Not all constraint subclasses must have this attribute, see __init__ note. Changing this attribute at runtime may result in “bad bounces”, see above.

class Rectangle(Constraint): A rectangular constraint. Size corresponds to side length.

class ModuloRectangle(Constraint): Like Rectangle, but values overflow, bounce is ignored.

class Ellipse(Constraint): An elliptic constraint. Size corresponds to radii.

st3m.ui.widgets module

Basic Usage

Sensor Widgets

Captouch Widgets

Single-Petal Widgets

Multi-Petal Widgets

Constraints

`st3m.ui.widgets` module