Here at balena, we understand that the primary users of our platform are engineers. For a long time, the tooling we provided for developers and DevOps engineers wasn’t as user-friendly or cohesive as we’d have liked it to be. So a few months back, we decided to focus on improving the experience when using balenaCloud and balenaOS. Here’s what we did…
- Introduced Local Mode
- Created Livepush
- Added Log Commands
- Refreshed SSH Commands
- Added Native installers for MacOS and Windows
With all the new features and functionality, we naturally had to make some significant behavioral changes to the CLI, for this reason, we decided to designate this as a major release. We believe most of the major changes work to improve the consistency and usability of the CLI and further our goal of having the CLI as a central tool in all balena workflows. See Major Changes for an in-depth list of all the changes in balena CLI v11, or if you just want to try it now click the button below to find the latest relase on GitHub.
Latest balena CLI on GitHub balena-cli on npm
Local mode
The first step in our mission to deliver a better user experience was finding a way to reduce the time between code changing and it running on your device. We already had the command balena local push, but this feature lived in isolation, it didn’t support multi-container, and was difficult to update. We decided to scrap the existing workflow and instead use the supervisor, as the container orchestration code already existed and now the supervisor was part of balenaOS in both managed and unmanaged devices. Note that local mode is only supported on .dev variants of balenaOS and on balena cloud it needs to be explicitly enabled.
To perform a local mode push, in your project directory, execute balena push <device-ip> or balena push <short-uuid>.local. This will attempt to communicate with the supervisor and docker daemon on the target device, which will detect information about your device, and resolve any Dockerfile.template files. The builds will then occur on-device. Note, this may be slower for the first push than using the cloud builder (in general), but subsequent pushes will make use of docker layer caching, and will be much faster.
Once the builds have completed, a target state is generated, in much the same way as balenaCloud. The supervisor then applies this target state, and your code will be running!
Two things then happen;
- The logs of the services and the supervisor will be streamed into your terminal. This can be controlled with the
--detachedflag. You can also specify which services you want to get logs from (--service) and whether to receive system (supervisor) logs (--system). - The directory in which the code resides will be watched for changes. When any change occurs, a livepush process (more on this below) will start, and your changes will be automatically applied. This can be turned off by supplying the
--noliveflag.
Livepush
Livepush is a recent innovation here at balena, created to streamline the entire workflow when developing on balena devices. At the core of livepush is the idea that it shouldn’t be necessary to change your workflow or code to use it, and we’ve tried to stick to that as much as possible.
As an example, let us consider the following fairly typical Dockerfile.template for a typescript application:
“`Dockerfile
FROM balenalib/%%BALENA_ARCH%%-node:10 # step 1
WORKDIR /usr/src/app #step 2
COPY package.json . #step 3
RUN npm install # step 4
COPY src/*.ts src/ # step 5
RUN npm run build # step 6
CMD node build/app.js # step 7
“`
The livepush process will look at the Dockerfile, and split it into groups internally. These groups have dependencies on external files, just like in Docker layer caching. If, for example, we changed the package.json file, step 3 onwards would become invalidated, but a source file change would only invalidate step 5 onwards.
The difference between docker layer caching and livepush is that once a layer is invalidated in docker, the entire layer needs to be regenerated, but this is not the case with livepush. To improve on docker layer caching, instead of creating a new image and container with every code change, we execute the dockerfile commands from within the running container. This means that, for example, if you added a dependency to your package.json, rather than having to install all of the dependencies again, only the new dependency would be. If your language stack supports incremental compilation, a source file change would cause only a single file to be recompiled.
Let’s use the above Dockerfile in an example. I add a dependency to the package.json file and hit save. Livepush detects this change and looks at the group information it’s holding internally. It discovers that step 3 references the changed file and copies the package.json into the container, overwriting the existing one. Now all other steps must be run (technically), so we re-run the npm install. Of course, all of the other dependencies are there, so only the new one will be installed (so we’re already quicker than docker would be).
Livepush then moves to step 5, discovers that the file changed is not referenced in this step, and skips it. Step 6 now has to be run, but due to typescript’s new incremental compilation no compilation would occur and we would quickly finish the process. At this point, the container will be restarted, so the CMD line can be re-executed, and the new code is running.
If we instead changed a source file, livepush would copy in the file at step 5, re-run the build (remember only a single file will be compiled) and the container would then be restarted.
The following shows a livepush process when I make a change to the frontend service:
Logs
I mentioned earlier that balena push now streams back device logs. It’s also possible to do this using the balena logs command, which accepts the same flags (--service, --system). This now works with multi-container and more importantly, the same command (balena logs) will work on a local device (if a <short-uuid>.local or <device-ip> is provided) or a cloud device (if a uuid is provided). It’s that easy!
SSH
The ssh commands have also been refreshed. Firstly balena local ssh has been removed, and balena ssh options have been revamped. The command now takes an application name, a device uuid or a local device address. Optionally it can also take a service name. If there is no service name, a shell on the host OS will be started, if there is a service name, a shell inside the service container will be started. If an application name is passed to balena ssh you will be presented with a dialogue to select the online device to connect to. Also, no more sudo!
Native Installers
Last but not least, we have a BETA release for our CLI native installers. Currently, there are installers for macOS and Windows, with Linux in the works. The installers offer a familiar click-through install experience for the CLI which improves and simplifies the installation process. No more npm install or messing with your OS path. You can find the installers on the CLI github releases page starting from v11.0.0 . In the coming weeks we will be improving these installers by adding easy update functionality and digitally signing them so MacOS and Windows don’t complain of “Unknown Developer”. We are excited by how much this simplifies the use of the CLI, and we can’t wait for your feedback on the new install experience.
Major Changes
To improve the consistency of the CLI, all the local namespace commands were removed or merged into top-level commands, with the exception of local flash and local configure.
balena local pushhas been replaced bybalena push <deviceIP | short_UUID.local>balena local sshhas been replaced bybalena ssh <deviceIP | short_UUID.local>balena local stophas been removedbalena local scanhas been moved tobalena scan
The behavior of balena ssh has subtly changed. It now requires that you specify the name of the service you wish to SSH into. If no service name is supplied, you will be dropped into a shell session on the balena HostOS.
balena push now defaults to using the livepush functionality. If you wish to maintain the old behavior you will need to supply the --nolive flag.
balena sync has been removed in favor of balena push as the livepush syncing offers a better and more consistent development workflow. In future versions we will also enable remote livepush to .dev devices.
Last of all, we have removed the balena signup command, all signups for balenaCloud will now be directed through dashboard.balena-cloud.com.
Get in Touch
There are many more exciting updates in the pipeline but in the meantime, we hope you enjoy the improved user experience we’re aiming to deliver. We would love to know what you think of these latest developments and answer any questions you may have. As always, you can find us in the forums!
