Programming

Once you feel some familiary with the REPL, you’re ready to advance to the next chapter: writing full-fledged applications that can draw graphics on the screen, respond to input and play sound!

Basics

Implementing a responsive user interface on a resource constrained device which at the same time should also output glitch free audio is not the easiest task in the world. The flow3r application programming environment tries make it a bit easier for you.

There are two major components to the running an app on the flower: the Reactor and at least one or more Responder s. The Reactor is a component which comes with the flow3r and takes care of all the heavy lifting for you. It decides when it is time to draw something on the display and it also gathers the data from a whole bunch of inputs like captouch or the buttons for you to work with.

A responder is a software component which can get called by the Reactor and is responsible to react to the input data and when asked draw something to the screen.

Example 1a: Display something

Let’s have a look at a very simple example involving a responder:

from st3m.reactor import Responder
import st3m.run

class Example(Responder):
    def __init__(self) -> None:
        pass

    def draw(self, ctx: Context) -> None:
        # Paint the background black
        ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()

        # Paint a red square in the middle of the display
        ctx.rgb(1, 0, 0).rectangle(-20, -20, 40, 40).fill()

    def think(self, ins: InputState, delta_ms: int) -> None:
        pass


st3m.run.run_responder(Example())

You can save this example as a Python file (e.g. example.py) and run it using mpremote run example.py. It should display a red square in the middle of the display and do nothing else.

You might already be able to guess the meaning of the three things that a responder has to implement:

Function	Meaning
__init__()	Called once before any of the other methods is run.
draw()	Called each time the display should be drawn.
think()	Called regularly with the latest input and sensor readings

It’s important to note that none of these methods is allowed take a significant amount of time if you want the user interface of the flow3r to feel snappy. You also need to make sure that each time draw() is called, everything you want to show is drawn again. Otherwise you will experience strange flickering or other artifacts on the screen.

Example 1b: React to input

If we want to react to the user, we can use the InputState which got handed to us. In this example we look at the state of the app (by default left) shoulder button. The values for buttons contained in the input state are one of InputButtonState.PRESSED_LEFT, PRESSED_RIGHT, PRESSED_DOWN, NOT_PRESSED - same values as in the low-level sys_buttons.

from st3m.reactor import Responder
import st3m.run

class Example(Responder):
    def __init__(self) -> None:
        self._x = -20

    def draw(self, ctx: Context) -> None:
        # Paint the background black
        ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()

        # Paint a red square in the middle of the display
        ctx.rgb(1, 0, 0).rectangle(self._x, -20, 40, 40).fill()

    def think(self, ins: InputState, delta_ms: int) -> None:
        direction = ins.buttons.app

        if direction == ins.buttons.PRESSED_LEFT:
            self._x -= 1
        elif direction == ins.buttons.PRESSED_RIGHT:
            self._x += 1


st3m.run.run_responder(Example())

Try it: when you run this code, you can move the red square using the app (by default left) shoulder button.

Example 1c: Taking time into consideration

The previous example moved the square around, but could you tell how fast it moved across the screen? What if you wanted it to move exactly 20 pixels per second to the left and 20 pixels per second to the right?

The think() method has an additional parameter we can use for this: delta_ms. It represents the time which has passed since the last call to think().

from st3m.reactor import Responder
import st3m.run

class Example(Responder):
    def __init__(self) -> None:
        self._x = -20.

    def draw(self, ctx: Context) -> None:
        # Paint the background black
        ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()

        # Paint a red square in the middle of the display
        ctx.rgb(1, 0, 0).rectangle(self._x, -20, 40, 40).fill()

    def think(self, ins: InputState, delta_ms: int) -> None:
        direction = ins.buttons.app # -1 (left), 1 (right), or 2 (pressed)

        if direction == ins.buttons.PRESSED_LEFT:
            self._x -= 20 * delta_ms / 1000
        elif direction == ins.buttons.PRESSED_RIGHT:
            self._x += 40 * delta_ms / 1000


st3m.run.run_responder(Example())

This becomes important if you need exact timings in your application, as the Reactor makes no explicit guarantee about how often think() will be called. Currently we are shooting for once every 20 milliseconds, but if something in the system takes a bit longer to process something, this number can change from one call to the next.

Example 1d: Automatic input processing

Working on the bare state of the buttons and the captouch petals can be cumbersome and error prone. the flow3r application framework gives you a bit of help in the form of the InputController which processes an input state and gives you higher level information about what is happening.

The InputController contains multiple Pressable sub-objects, for example the app/OS buttons are available as following attributes on the InputController:

Attribute on `InputControlller`	Meaning
`.buttons.app.left`	App button, pushed left
`.buttons.app.middle`	App button, pushed down
`.buttons.app.right`	App button, pushed right
`.buttons.os.left`	OS button, pushed left
`.buttons.os.middle`	OS button, pushed down
`.buttons.os.right`	OS button, pushed right

And each Pressable in turn contains the following attributes, all of which are valid within the context of a single think() call:

Attribute on `Pressable`	Meaning
`.pressed`	Button has just started being pressed, ie. it’s a Half Press down.
`.down`	Button is being held down.
`.released`	Button has just stopped being pressed, ie. it’s a Half Press up.
`.up`	Button is not being held down.

The following example shows how to properly react to single button presses without having to think about what happens if the user presses the button for a long time. It uses the InputController to detect single button presses and switches between showing a circle (by drawing a 360 deg arc) and a square.

from st3m.reactor import Responder
from st3m.input import InputController
from st3m.utils import tau

import st3m.run

class Example(Responder):
    def __init__(self) -> None:
        self.input = InputController()
        self._x = -20.
        self._draw_rectangle = True

    def draw(self, ctx: Context) -> None:
        # Paint the background black
        ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()

        # Paint a red square in the middle of the display
        if self._draw_rectangle:
            ctx.rgb(1, 0, 0).rectangle(self._x, -20, 40, 40).fill()
        else:
            ctx.rgb(1, 0, 0).arc(self._x, -20, 40, 0, tau, 0).fill()

    def think(self, ins: InputState, delta_ms: int) -> None:
        self.input.think(ins, delta_ms) # let the input controller to its magic

        if self.input.buttons.app.middle.pressed:
            self._draw_rectangle = not self._draw_rectangle

        if self.input.buttons.app.left.pressed:
            self._x -= 20 * delta_ms / 1000
        elif self.input.buttons.app.right.pressed:
            self._x += 40 * delta_ms / 1000


st3m.run.run_responder(Example())

Managing multiple views

If you want to write a more advanced application you probably also want to display more than one screen (or view as we call them). With just the Responder class this can become a bit tricky as it never knows when it is visible and when it is not. It also doesn’t directly allow you to launch a new screen.

To help you with that you can use a View instead. It can tell you when it becomes visible, when it is about to become inactive (invisible) and you can also use it to bring a new screen or widget into the foreground or remove it again from the screen.

Example 2a: Managing two views

In this example we use a basic View to switch between to different screens using a button. One screen shows a red square, the other one a green square. You can of course put any kind of complex processing into the two different views. We make use of an InputController again to handle the button presses.

from st3m.input import InputController
from st3m.ui.view import View
import st3m.run

class SecondScreen(View):
    def __init__(self) -> None:
        self.input = InputController()
        self._vm = None

    def on_enter(self, vm: Optional[ViewManager]) -> None:
        self._vm = vm

        # Ignore the button which brought us here until it is released
        self.input._ignore_pressed()

    def draw(self, ctx: Context) -> None:
        # Paint the background black
        ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()
        # Green square
        ctx.rgb(0, 1, 0).rectangle(-20, -20, 40, 40).fill()

    def think(self, ins: InputState, delta_ms: int) -> None:
        self.input.think(ins, delta_ms) # let the input controller to its magic

        # No need to handle returning back to Example on button press - the
        # flow3r's ViewManager takes care of that automatically.


class Example(View):
    def __init__(self) -> None:
        self.input = InputController()
        self._vm = None

    def draw(self, ctx: Context) -> None:
        # Paint the background black
        ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()
        # Red square
        ctx.rgb(1, 0, 0).rectangle(-20, -20, 40, 40).fill()


    def on_enter(self, vm: Optional[ViewManager]) -> None:
        self._vm = vm
        self.input._ignore_pressed()

    def think(self, ins: InputState, delta_ms: int) -> None:
        self.input.think(ins, delta_ms) # let the input controller to its magic

        if self.input.buttons.app.middle.pressed:
            self._vm.push(SecondScreen())

st3m.run.run_view(Example())

Try it using mpremote. The OS shoulder button (right shoulder unless swapped in settings) switches between the two views. To avoid that the still pressed button immediately closes SecondScreen we make us of a special method of the InputController which hides the pressed button from the view until it is released again.

Note

Pressing the OS shoulder button in REPL mode will currently reset the badge.

Until this is fixed, you can test view switching by copying the app to your badge and running from the menu.

Example 2b: Easier view management

The above code is so universal that we provide a special view which takes care of this boilerplate: BaseView. It integrated a local InputController on self.input and a copy of the ViewManager which caused the View to enter on self.vm.

Here is our previous example rewritten to make use of BaseView:

from st3m.ui.view import BaseView
import st3m.run

class SecondScreen(BaseView):
    def __init__(self) -> None:
        # Remember to call super().__init__() if you implement your own
        # constructor!
        super().__init__()

    def on_enter(self, vm: Optional[ViewManager]) -> None:
        # Remember to call super().on_enter() if you implement your own
        # on_enter!
        super().on_enter(vm)

    def draw(self, ctx: Context) -> None:
        # Paint the background black
        ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()
        # Green square
        ctx.rgb(0, 1, 0).rectangle(-20, -20, 40, 40).fill()

class Example(BaseView):
    def draw(self, ctx: Context) -> None:
        # Paint the background black
        ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()
        # Red square
        ctx.rgb(1, 0, 0).rectangle(-20, -20, 40, 40).fill()

    def think(self, ins: InputState, delta_ms: int) -> None:
        super().think(ins, delta_ms) # Let BaseView do its thing

        if self.input.buttons.app.middle.pressed:
            self.vm.push(SecondScreen())

st3m.run.run_view(Example())

In some cases, it’s useful to know more about the view’s lifecycle, so tasks like long-blocking loading screens can be synchronized with view transition animations. You’ll also be interested in those events when doing unusual things with rendering, such as partial redraws or direct framebuffer manipulation.

Here’s the list of methods that you can implement in your class:

Function	Meaning
`.on_enter(vm)`	The view has became active and is about to start receiving `.think()` and `.draw()` calls.
`.on_enter_done()`	The view transition has finished animating; the whole screen is now under the control of the view. If you want to perform a long running blocking task (such as loading audio samples), it’s a good idea to initiate it here to not block during the transition animation.
`.on_exit()`	The view has became inactive and started to transition out; no further `.think()` calls will be made until the next `.on_enter()` unless you return `True` from this handler. Do the clean-up of shared resources (such as LEDs or the media engine) here. This way you won’t step onto the next view’s shoes, as it may want to use them in its `.on_enter()` already. Note: When returning `True`, make sure not to handle input events not meant for this view in your `.think()` handler during transition animation (see `.is_active()` below).
`.on_exit_done()`	The view transition has finished animating; no further `.draw()` or `.think()` calls will be made until the next `.on_enter()`. A good place to do the clean-up of resources you have exclusive control over, such as bl00mbox channels.
`.show_icons()`	Return `True` from it to have indicator icons drawn on top of your view by the system, just like in the main menu. Defaults to `False`.

ViewManager also provides some methods to make handling common cases easier:

Function / property

Meaning

.is_active(view)

Returns a bool indicating whether the passed view is currently active. The active view is the one that’s expected to be in control of user’s input and the view stack. A view becomes active at .on_enter() and stops being active at .on_exit().

.transitioning

Whether a transition animation is currently in progress.

.direction

Returns the direction in which the currently active view has became one:

ViewTransitionDirection.NONE if it has replaced another view;

ViewTransitionDirection.FORWARD if it was pushed into the stack;

ViewTransitionDirection.BACKWARD if another view was popped.

On top of that, BaseView implements an additional helper method .is_active(), which is simply a bit less awkward way to call self.vm.is_active(self).

Writing an application for the menu system

All fine and good, you were able to write an application that you can run with mpremote, but certainly you also want to run it from flow3r’s menu system.

Let’s introduce the final class you should actually be using for application development: Application. It builds upon BaseView (so you still have access to self.input and self.vm) but additionally is made aware of an ApplicationContext on startup and can be registered into a menu.

Here is our previous code changed to use Application for the base of its main view:

from st3m.application import Application, ApplicationContext
from st3m.ui.view import BaseView, ViewManager
from st3m.input import InputState
from ctx import Context
import st3m.run

class SecondScreen(BaseView):
    def draw(self, ctx: Context) -> None:
        # Paint the background black
        ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()
        # Green square
        ctx.rgb(0, 1, 0).rectangle(-20, -20, 40, 40).fill()

class MyDemo(Application):
    def __init__(self, app_ctx: ApplicationContext) -> None:
        super().__init__(app_ctx)
        # Ignore the app_ctx for now.

    def draw(self, ctx: Context) -> None:
        # Paint the background black
        ctx.rgb(0, 0, 0).rectangle(-120, -120, 240, 240).fill()
        # Red square
        ctx.rgb(1, 0, 0).rectangle(-20, -20, 40, 40).fill()

    def think(self, ins: InputState, delta_ms: int) -> None:
        super().think(ins, delta_ms) # Let Application do its thing

        if self.input.buttons.app.middle.pressed:
            self.vm.push(SecondScreen())

if __name__ == '__main__':
    # Continue to make runnable via mpremote run.
    st3m.run.run_app(MyDemo)

To add the application to the menu we are missing one more thing: a flow3r.toml file which describes the application so flow3r knows where to put it in the menu system. Together with the Python code this file forms a so called bundle (see also BundleMetadata).

[app]
name = "My Demo"
category = "Apps"

[entry]
class = "MyDemo"

[metadata]
author = "You :)"
license = "pick one, LGPL/MIT maybe?"
url = "https://git.flow3r.garden/you/mydemo"

Save this as flow3r.toml together with the Python code as __init__.py in a folder (name doesn’t matter) and put that folder into one of the possible application directories (see below) using `Disk Mode`_. Restart the flow3r and it should pick up your new application.

Medium	Path in Disk Mode	Path on Badge	Notes
Flash	`sys/apps`	`/flash/sys/apps`	“Default” apps.
Flash	`apps`	`/flash/apps`	Doesn’t exist by default. Split from `sys` to allow for cleaner updates.
SD	`apps`	`/sd/apps`	Doesn’t exist by default. Will be retained even across badge reflashes.

Handling assets

Using Application also gives you access to the ApplicationContext as self.app_ctx, which for example gives you a way to find out the base path of your app in app_ctx.bundle_path or its bundle metadata in app_ctx.bundle_metadata. It’s very important not to hardcode paths to your assets and use bundle_path instead, because applications can be installed in various places depending on installation method and moved between internal flash and SD card.

A simple app that does nothing but draws a file named image.png from its directory could look like this:

from st3m.application import Application
from ctx import Context
import st3m.run

class MyDemo(Application):
    def draw(self, ctx: Context) -> None:
        # Draw a image file
        ctx.image(f"{self.app_ctx.bundle_path}/image.png", -120, -120, 240, 240)

if __name__ == '__main__':
    # Continue to make runnable via mpremote run.
    st3m.run.run_app(MyDemo, "/flash/apps/MyDemo")

Note the default path provided in st3m.run.run_app call - this is the path that’s going to be used when running standalone via mpremote, so set it to the path where you’re putting your app files on your badge during development.